The flow of ideas in word embeddings
- URL: http://arxiv.org/abs/2307.16819v1
- Date: Wed, 26 Jul 2023 15:51:31 GMT
- Title: The flow of ideas in word embeddings
- Authors: Debayan Dasgupta
- Abstract summary: Flow of ideas has been extensively studied by physicists, psychologists, and machine learning engineers.
This paper adopts specific tools from microrheology to investigate the similarity-based flow of ideas.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The flow of ideas has been extensively studied by physicists, psychologists,
and machine learning engineers. This paper adopts specific tools from
microrheology to investigate the similarity-based flow of ideas. We introduce a
random walker in word embeddings and study its behavior. Such
similarity-mediated random walks through the embedding space show signatures of
anomalous diffusion commonly observed in complex structured systems such as
biological cells and complex fluids. The paper concludes by proposing the
application of popular tools employed in the study of random walks and
diffusion of particles under Brownian motion to assess quantitatively the
incorporation of diverse ideas in a document. Overall, this paper presents a
self-referenced method combining microrheology and machine learning concepts to
explore the meandering tendencies of language models and their potential
association with creativity.
Related papers
- Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
Ontologies are widely used for representing domain knowledge and meta data.
One straightforward solution is to integrate statistical analysis and machine learning.
Numerous papers have been published on embedding, but a lack of systematic reviews hinders researchers from gaining a comprehensive understanding of this field.
arXiv Detail & Related papers (2024-06-16T14:49:19Z) - Sifting through the Noise: A Survey of Diffusion Probabilistic Models and Their Applications to Biomolecules [0.7366405857677227]
Diffusion probabilistic models have made their way into a number of high-profile applications.
This paper serves as a general overview for the theory behind these models and the current state of research.
arXiv Detail & Related papers (2024-05-31T21:39:51Z) - Language Evolution with Deep Learning [49.879239655532324]
Computational modeling plays an essential role in the study of language emergence.
It aims to simulate the conditions and learning processes that could trigger the emergence of a structured language.
This chapter explores another class of computational models that have recently revolutionized the field of machine learning: deep learning models.
arXiv Detail & Related papers (2024-03-18T16:52:54Z) - Seeing Unseen: Discover Novel Biomedical Concepts via
Geometry-Constrained Probabilistic Modeling [53.7117640028211]
We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues.
We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space.
A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
arXiv Detail & Related papers (2024-03-02T00:56:05Z) - Brain-Inspired Machine Intelligence: A Survey of
Neurobiologically-Plausible Credit Assignment [65.268245109828]
We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology.
We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors.
The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
arXiv Detail & Related papers (2023-12-01T05:20:57Z) - Discovering mesoscopic descriptions of collective movement with neural
stochastic modelling [4.7163839266526315]
Collective motion at small to medium group sizes ($sim$10-1000 individuals, also called the meso') can show nontrivial features due to order.
Here, we use a physics-inspired, network based approach to characterize the neural group dynamics of interacting individuals.
We apply this technique on both synthetic and real-world datasets, and identify the deterministic and aspects of the dynamics using drift and diffusion fields.
arXiv Detail & Related papers (2023-03-17T11:49:17Z) - Computing with Categories in Machine Learning [1.7679374058425343]
We introduce DisCoPyro as a categorical structure learning framework.
DisCoPyro combines categorical structures with amortized variational inference.
We speculate that DisCoPyro could ultimately contribute to the development of artificial general intelligence.
arXiv Detail & Related papers (2023-03-07T17:26:18Z) - Foundations and Recent Trends in Multimodal Machine Learning:
Principles, Challenges, and Open Questions [68.6358773622615]
This paper provides an overview of the computational and theoretical foundations of multimodal machine learning.
We propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification.
Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches.
arXiv Detail & Related papers (2022-09-07T19:21:19Z) - Semantic Search for Large Scale Clinical Ontologies [63.71950996116403]
We present a deep learning approach to build a search system for large clinical vocabularies.
We propose a Triplet-BERT model and a method that generates training data based on semantic training data.
The model is evaluated using five real benchmark data sets and the results show that our approach achieves high results on both free text to concept and concept to searching concept vocabularies.
arXiv Detail & Related papers (2022-01-01T05:15:42Z) - Hierarchically Organized Latent Modules for Exploratory Search in
Morphogenetic Systems [21.23182328329019]
We introduce a novel dynamic and modular architecture that enables unsupervised learning of a hierarchy of diverse representations.
We show that this system forms a discovery assistant that can efficiently adapt its diversity search towards preferences of a user.
arXiv Detail & Related papers (2020-07-02T15:28:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.