A simplicity bubble problem and zemblanity in digitally intermediated
societies
- URL: http://arxiv.org/abs/2304.10681v2
- Date: Sat, 13 Jan 2024 20:33:16 GMT
- Title: A simplicity bubble problem and zemblanity in digitally intermediated
societies
- Authors: Felipe S. Abrah\~ao, Ricardo P. Cavassane, Michael Winter, Mariana
Vitti Rodrigues, Itala M. L. D'Ottaviano
- Abstract summary: We discuss the ubiquity of Big Data and machine learning in society.
We show that there is a ceiling above which formal knowledge cannot further decrease the probability of zemblanitous findings.
- Score: 1.54280001017091
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this article, we discuss the ubiquity of Big Data and machine learning in
society and propose that it evinces the need of further investigation of their
fundamental limitations. We extend the "too much information tends to behave
like very little information" phenomenon to formal knowledge about lawlike
universes and arbitrary collections of computably generated datasets. This
gives rise to the simplicity bubble problem, which refers to a learning
algorithm equipped with a formal theory that can be deceived by a dataset to
find a locally optimal model which it deems to be the global one. In the
context of lawlike (computable) universes and formal learning systems, we show
that there is a ceiling above which formal knowledge cannot further decrease
the probability of zemblanitous findings, should the randomly generated data
made available to the formal learning system be sufficiently large in
comparison to their joint complexity. We also argue that this is an
epistemological limitation that may generate unpredictable problems in
digitally intermediated societies.
Related papers
- Open-world Machine Learning: A Review and New Outlooks [83.6401132743407]
This paper aims to provide a comprehensive introduction to the emerging open-world machine learning paradigm.
It aims to help researchers build more powerful AI systems in their respective fields, and to promote the development of artificial general intelligence.
arXiv Detail & Related papers (2024-03-04T06:25:26Z) - The Computational Complexity of Concise Hypersphere Classification [49.57441416941195]
This paper is the first complexity-theoretic study of the hypersphere classification problem for binary data.
We use the fine-grained parameterized complexity paradigm to analyze the impact of structural properties that may be present in the input data.
arXiv Detail & Related papers (2023-12-12T09:33:03Z) - The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning [80.1018596899899]
We argue that neural network models share this same preference, formalized using Kolmogorov complexity.
Our experiments show that pre-trained and even randomly language models prefer to generate low-complexity sequences.
These observations justify the trend in deep learning of unifying seemingly disparate problems with an increasingly small set of machine learning models.
arXiv Detail & Related papers (2023-04-11T17:22:22Z) - Principled Knowledge Extrapolation with GANs [92.62635018136476]
We study counterfactual synthesis from a new perspective of knowledge extrapolation.
We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem.
Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
arXiv Detail & Related papers (2022-05-21T08:39:42Z) - Causal Reasoning Meets Visual Representation Learning: A Prospective
Study [117.08431221482638]
Lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models.
Inspired by the strong inference ability of human-level agents, recent years have witnessed great effort in developing causal reasoning paradigms.
This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods.
arXiv Detail & Related papers (2022-04-26T02:22:28Z) - A Simplicity Bubble Problem in Formal-Theoretic Learning Systems [1.7996150751268578]
We show that current approaches to machine learning can always be deceived, naturally or artificially, by sufficiently large datasets.
We discuss the framework and additional empirical conditions to be met in order to circumvent this deceptive phenomenon.
arXiv Detail & Related papers (2021-12-22T23:44:47Z) - Topologically Regularized Data Embeddings [22.222311627054875]
We introduce a new set of topological losses, and propose their usage as a way for topologically regularizing data embeddings to naturally represent a prespecified model.
We include experiments on synthetic and real data that highlight the usefulness and versatility of this approach.
arXiv Detail & Related papers (2021-10-18T11:25:47Z) - Parsimonious Inference [0.0]
Parsimonious inference is an information-theoretic formulation of inference over arbitrary architectures.
Our approaches combine efficient encodings with prudent sampling strategies to construct predictive ensembles without cross-validation.
arXiv Detail & Related papers (2021-03-03T04:13:14Z) - When is Memorization of Irrelevant Training Data Necessary for
High-Accuracy Learning? [53.523017945443115]
We describe natural prediction problems in which every sufficiently accurate training algorithm must encode, in the prediction model, essentially all the information about a large subset of its training examples.
Our results do not depend on the training algorithm or the class of models used for learning.
arXiv Detail & Related papers (2020-12-11T15:25:14Z) - Random thoughts about Complexity, Data and Models [0.0]
Data Science and Machine learning have been growing strong for the past decade.
We investigate the subtle relation between "data and models"
Key issue for appraisal of relation between algorithmic complexity and algorithmic learning is to do with concepts of compressibility, determinism and predictability.
arXiv Detail & Related papers (2020-04-16T14:27:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.