Creolizing the Web
- URL: http://arxiv.org/abs/2102.12382v1
- Date: Wed, 24 Feb 2021 16:08:45 GMT
- Title: Creolizing the Web
- Authors: Abhinav Tamaskar, Roy Rinberg, Sunandan Chakraborty, Bud Mishra
- Abstract summary: We present a method for detecting evolutionary patterns in a sociological model of language evolution.
We develop a minimalistic model that provides a rigorous base for any generalized evolutionary model for language based on communication between individuals.
We present empirical results and their interpretations on a real world dataset from rdt to identify communities and echo chambers for opinions.
- Score: 2.393911349115195
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The evolution of language has been a hotly debated subject with contradicting
hypotheses and unreliable claims. Drawing from signalling games, dynamic
population mechanics, machine learning and algebraic topology, we present a
method for detecting evolutionary patterns in a sociological model of language
evolution. We develop a minimalistic model that provides a rigorous base for
any generalized evolutionary model for language based on communication between
individuals. We also discuss theoretical guarantees of this model, ranging from
stability of language representations to fast convergence of language by
temporal communication and language drift in an interactive setting. Further we
present empirical results and their interpretations on a real world dataset
from \rdt to identify communities and echo chambers for opinions, thus placing
obstructions to reliable communication among communities.
Related papers
- NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication [2.184775414778289]
Recently introduced NeLLCom framework allows agents to first learn an artificial language and then use it to communicate.
We extend this framework by introducing more realistic role-alternating agents and group communication.
arXiv Detail & Related papers (2024-07-19T03:03:21Z) - Modeling language contact with the Iterated Learning Model [0.0]
Iterated learning models are agent-based models of language change.
A recently introduced type of iterated learning model, the Semi-Supervised ILM is used to simulate language contact.
arXiv Detail & Related papers (2024-06-11T01:43:23Z) - Emergent communication and learning pressures in language models: a language evolution perspective [5.371337604556311]
We find that the emergent communication literature excels at designing and adapting models to recover initially absent linguistic phenomena of natural languages.
We identify key pressures that have recovered initially absent human patterns in emergent communication models.
This may serve as inspiration for how to design language models for language acquisition and language evolution research.
arXiv Detail & Related papers (2024-03-21T14:33:34Z) - Language Evolution with Deep Learning [49.879239655532324]
Computational modeling plays an essential role in the study of language emergence.
It aims to simulate the conditions and learning processes that could trigger the emergence of a structured language.
This chapter explores another class of computational models that have recently revolutionized the field of machine learning: deep learning models.
arXiv Detail & Related papers (2024-03-18T16:52:54Z) - From Word Models to World Models: Translating from Natural Language to
the Probabilistic Language of Thought [124.40905824051079]
We propose rational meaning construction, a computational framework for language-informed thinking.
We frame linguistic meaning as a context-sensitive mapping from natural language into a probabilistic language of thought.
We show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings.
We extend our framework to integrate cognitively-motivated symbolic modules.
arXiv Detail & Related papers (2023-06-22T05:14:00Z) - Commonsense Knowledge Transfer for Pre-trained Language Models [83.01121484432801]
We introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model.
It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model.
It then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction.
arXiv Detail & Related papers (2023-06-04T15:44:51Z) - Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions.
Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
arXiv Detail & Related papers (2022-10-14T02:35:19Z) - Language Model Evaluation Beyond Perplexity [47.268323020210175]
We analyze whether text generated from language models exhibits the statistical tendencies present in the human-generated text on which they were trained.
We find that neural language models appear to learn only a subset of the tendencies considered, but align much more closely with empirical trends than proposed theoretical distributions.
arXiv Detail & Related papers (2021-05-31T20:13:44Z) - The Rediscovery Hypothesis: Language Models Need to Meet Linguistics [8.293055016429863]
We study whether linguistic knowledge is a necessary condition for good performance of modern language models.
We show that language models that are significantly compressed but perform well on their pretraining objectives retain good scores when probed for linguistic structures.
This result supports the rediscovery hypothesis and leads to the second contribution of our paper: an information-theoretic framework that relates language modeling objective with linguistic information.
arXiv Detail & Related papers (2021-03-02T15:57:39Z) - Experience Grounds Language [185.73483760454454]
Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates.
Despite the incredible effectiveness of language processing models to tackle tasks after being trained on text alone, successful linguistic communication relies on a shared experience of the world.
arXiv Detail & Related papers (2020-04-21T16:56:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.