Enforcing and Discovering Structure in Machine Learning

• URL: http://arxiv.org/abs/2111.13693v1
• Date: Fri, 26 Nov 2021 15:34:55 GMT
• Title: Enforcing and Discovering Structure in Machine Learning
• Authors: Francesco Locatello
• Abstract summary: It may be prudent to enforce corresponding structural properties to a learning algorithm's solution. In this dissertation, we consider two different research areas that concern structuring a learning algorithm's solution.
• Score: 18.750116414606698
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The world is structured in countless ways. It may be prudent to enforce corresponding structural properties to a learning algorithm's solution, such as incorporating prior beliefs, natural constraints, or causal structures. Doing so may translate to faster, more accurate, and more flexible models, which may directly relate to real-world impact. In this dissertation, we consider two different research areas that concern structuring a learning algorithm's solution: when the structure is known and when it has to be discovered.

Related papers

• StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization [94.31508613367296]
  Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) We propose StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure. Experiments show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios.
  arXiv  Detail & Related papers  (2024-10-11T13:52:44Z)
• On the Role of Information Structure in Reinforcement Learning for Partially-Observable Sequential Teams and Games [55.2480439325792]
  In a sequential decision-making problem, the information structure is the description of how events in the system occurring at different points in time affect each other. By contrast, real-world sequential decision-making problems typically involve a complex and time-varying interdependence of system variables. We formalize a novel reinforcement learning model which explicitly represents the information structure.
  arXiv  Detail & Related papers  (2024-03-01T21:28:19Z)
• Causal discovery using dynamically requested knowledge [7.904709685523615]
  Causal Bayesian Networks (CBNs) are an important tool for reasoning under uncertainty in complex real-world systems. We investigate a novel approach where the structure learning algorithm itself dynamically identifies and requests knowledge for relationships that the algorithm identifies as uncertain. We show that it offers considerable gains in structural accuracy, which are generally larger than those offered by existing approaches for integrating knowledge.
  arXiv  Detail & Related papers  (2023-10-17T11:21:23Z)
• Physics of Language Models: Part 1, Learning Hierarchical Language Structures [51.68385617116854]
  Transformer-based language models are effective but complex, and understanding their inner workings is a significant challenge. We introduce a family of synthetic CFGs that produce hierarchical rules, capable of generating lengthy sentences. We demonstrate that generative models like GPT can accurately learn this CFG language and generate sentences based on it.
  arXiv  Detail & Related papers  (2023-05-23T04:28:16Z)
• Principled and Efficient Motif Finding for Structure Learning of Lifted Graphical Models [5.317624228510748]
  Structure learning is a core problem in AI central to the fields of neuro-symbolic AI and statistical relational learning. We present the first principled approach for mining structural motifs in lifted graphical models. We show that we outperform state-of-the-art structure learning approaches by up to 6% in terms of accuracy and up to 80% in terms of runtime.
  arXiv  Detail & Related papers  (2023-02-09T12:21:55Z)
• Differentiable and Transportable Structure Learning [73.84540901950616]
  We introduce D-Struct, which recovers transportability in the discovered structures through a novel architecture and loss function. Because D-Struct remains differentiable, our method can be easily adopted in existing differentiable architectures.
  arXiv  Detail & Related papers  (2022-06-13T17:50:53Z)
• Learning Probabilistic Structural Representation for Biomedical Image Segmentation [37.07198480786721]
  We propose the first deep learning method to learn a structural representation. We empirically demonstrate the strength of our method, i.e., generating true structures rather than pixel-maps with better topological integrity.
  arXiv  Detail & Related papers  (2022-06-03T06:00:26Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Learning Bayesian Networks in the Presence of Structural Side Information [22.734574764075226]
  We study the problem of learning a Bayesian network (BN) of a set of variables when structural side information about the system is available. We develop an algorithm that efficiently incorporates such knowledge into the learning process. As a consequence of our work, we show that bounded treewidth BNs can be learned with complexity.
  arXiv  Detail & Related papers  (2021-12-20T22:14:19Z)
• Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components [71.03032589756434]
  We investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. We characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.
  arXiv  Detail & Related papers  (2021-06-16T17:12:26Z)
• Towards Structured Prediction in Bioinformatics with Deep Learning [11.055292483959414]
  In bioinformatics, we often need to predict more complex structured targets, such as 2D images and 3D molecular structures. Here, we argue that the following ideas can help resolve structured prediction problems in bioinformatics. We demonstrate our ideas with six projects from four bioinformatics subfields.
  arXiv  Detail & Related papers  (2020-08-25T02:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.