Structure Learning via Mutual Information

• URL: http://arxiv.org/abs/2409.14235v1
• Date: Sat, 21 Sep 2024 19:33:56 GMT
• Title: Structure Learning via Mutual Information
• Authors: Jeremy Nixon,
• Abstract summary: We propose a framework for learning and representing functional relationships in data using mutual information (MI) features. Our method aims to capture the underlying structure of information in datasets, enabling more efficient and generalizable learning algorithms.
• Score: 0.8702432681310399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a novel approach to machine learning algorithm design based on information theory, specifically mutual information (MI). We propose a framework for learning and representing functional relationships in data using MI-based features. Our method aims to capture the underlying structure of information in datasets, enabling more efficient and generalizable learning algorithms. We demonstrate the efficacy of our approach through experiments on synthetic and real-world datasets, showing improved performance in tasks such as function classification, regression, and cross-dataset transfer. This work contributes to the growing field of metalearning and automated machine learning, offering a new perspective on how to leverage information theory for algorithm design and dataset analysis and proposing new mutual information theoretic foundations to learning algorithms.

Related papers

• Deep Learning and Machine Learning -- Object Detection and Semantic Segmentation: From Theory to Applications [17.571124565519263]
  Book covers state-of-the-art advancements in machine learning and deep learning. Focuses on convolutional neural networks (CNNs), YOLO architectures, and transformer-based approaches like DETR. Book also delves into the integration of artificial intelligence (AI) techniques and large language models for enhanced object detection.
  arXiv  Detail & Related papers  (2024-10-21T02:10:49Z)
• A Unified Framework for Neural Computation and Learning Over Time [56.44910327178975]
  Hamiltonian Learning is a novel unified framework for learning with neural networks "over time" It is based on differential equations that: (i) can be integrated without the need of external software solvers; (ii) generalize the well-established notion of gradient-based learning in feed-forward and recurrent networks; (iii) open to novel perspectives.
  arXiv  Detail & Related papers  (2024-09-18T14:57:13Z)
• Extraction of Research Objectives, Machine Learning Model Names, and Dataset Names from Academic Papers and Analysis of Their Interrelationships Using LLM and Network Analysis [0.0]
  This study proposes a methodology extracting tasks, machine learning methods, and dataset names from scientific papers. The proposed method's expression extraction performance, when using Llama3, achieves an F-score exceeding 0.8 across various categories. Benchmarking results on financial domain papers have demonstrated the effectiveness of this method.
  arXiv  Detail & Related papers  (2024-08-22T03:10:52Z)
• Benchmark on Drug Target Interaction Modeling from a Structure Perspective [48.60648369785105]
  Drug-target interaction prediction is crucial to drug discovery and design. Recent methods, such as those based on graph neural networks (GNNs) and Transformers, demonstrate exceptional performance across various datasets. We conduct a comprehensive survey and benchmark for drug-target interaction modeling from a structure perspective, via integrating tens of explicit (i.e., GNN-based) and implicit (i.e., Transformer-based) structure learning algorithms.
  arXiv  Detail & Related papers  (2024-07-04T16:56:59Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• An Entropy-Based Model for Hierarchical Learning [3.1473798197405944]
  A common feature among real-world datasets is that data domains are multiscale. We propose a learning model that exploits this multiscale data structure. The hierarchical learning model is inspired by the logical and progressive easy-to-hard learning mechanism of human beings.
  arXiv  Detail & Related papers  (2022-12-30T13:14:46Z)
• Latent Properties of Lifelong Learning Systems [59.50307752165016]
  We introduce an algorithm-agnostic explainable surrogate-modeling approach to estimate latent properties of lifelong learning algorithms. We validate the approach for estimating these properties via experiments on synthetic data.
  arXiv  Detail & Related papers  (2022-07-28T20:58:13Z)
• CateCom: a practical data-centric approach to categorization of computational models [77.34726150561087]
  We present an effort aimed at organizing the landscape of physics-based and data-driven computational models. We apply object-oriented design concepts and outline the foundations of an open-source collaborative framework.
  arXiv  Detail & Related papers  (2021-09-28T02:59:40Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• An Information-Theoretic Perspective on Overfitting and Underfitting [0.0]
  We present an information-theoretic framework for understanding overfitting and underfitting in machine learning. We prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.
  arXiv  Detail & Related papers  (2020-10-12T23:24:47Z)
• Clustering Analysis of Interactive Learning Activities Based on Improved BIRCH Algorithm [0.0]
  The construction of good learning behavior is of great significance to learners' learning process and learning effect, and is the key basis of data-driven education decision-making. It is necessary to obtain the online learning behavior big data set of multi period and multi course, and describe the learning behavior as multi-dimensional learning interaction activities. We design an improved algorithm of BIRCH clustering based on random walking strategy, which realizes the retrieval evaluation and data of key learning interaction activities.
  arXiv  Detail & Related papers  (2020-10-08T07:46:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.