Multiblock-Networks: A Neural Network Analog to Component Based Methods for Multi-Source Data

• URL: http://arxiv.org/abs/2109.10279v1
• Date: Tue, 21 Sep 2021 16:00:15 GMT
• Title: Multiblock-Networks: A Neural Network Analog to Component Based Methods for Multi-Source Data
• Authors: Anna Jenul and Stefan Schrunner and Runar Helin and Kristian Hovde Liland and Cecilia Marie Futs{\ae}ther and Oliver Tomic
• Abstract summary: We propose a setup to transfer the concepts of component based statistical models to neural network architectures. Thereby, we combine the flexibility of neural networks with the concepts for interpreting block relevance in multiblock methods. Our results underline that multiblock networks allow for basic model interpretation while matching the performance of ordinary feed-forward neural networks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training predictive models on datasets from multiple sources is a common, yet challenging setup in applied machine learning. Even though model interpretation has attracted more attention in recent years, many modeling approaches still focus mainly on performance. To further improve the interpretability of machine learning models, we suggest the adoption of concepts and tools from the well-established framework of component based multiblock analysis, also known as chemometrics. Nevertheless, artificial neural networks provide greater flexibility in model architecture and thus, often deliver superior predictive performance. In this study, we propose a setup to transfer the concepts of component based statistical models, including multiblock variants of principal component regression and partial least squares regression, to neural network architectures. Thereby, we combine the flexibility of neural networks with the concepts for interpreting block relevance in multiblock methods. In two use cases we demonstrate how the concept can be implemented in practice, and compare it to both common feed-forward neural networks without blocks, as well as statistical component based multiblock methods. Our results underline that multiblock networks allow for basic model interpretation while matching the performance of ordinary feed-forward neural networks.

Related papers

• Towards Scalable and Versatile Weight Space Learning [51.78426981947659]
  This paper introduces the SANE approach to weight-space learning. Our method extends the idea of hyper-representations towards sequential processing of subsets of neural network weights.
  arXiv  Detail & Related papers  (2024-06-14T13:12:07Z)
• BEND: Bagging Deep Learning Training Based on Efficient Neural Network Diffusion [56.9358325168226]
  We propose a Bagging deep learning training algorithm based on Efficient Neural network Diffusion (BEND) Our approach is simple but effective, first using multiple trained model weights and biases as inputs to train autoencoder and latent diffusion model. Our proposed BEND algorithm can consistently outperform the mean and median accuracies of both the original trained model and the diffused model.
  arXiv  Detail & Related papers  (2024-03-23T08:40:38Z)
• Dependency-based Mixture Language Models [53.152011258252315]
  We introduce the Dependency-based Mixture Language Models. In detail, we first train neural language models with a novel dependency modeling objective. We then formulate the next-token probability by mixing the previous dependency modeling probability distributions with self-attention.
  arXiv  Detail & Related papers  (2022-03-19T06:28:30Z)
• Connections between Numerical Algorithms for PDEs and Neural Networks [8.660429288575369]
  We investigate numerous structural connections between numerical algorithms for partial differential equations (PDEs) and neural networks. Our goal is to transfer the rich set of mathematical foundations from the world of PDEs to neural networks.
  arXiv  Detail & Related papers  (2021-07-30T16:42:45Z)
• Creating Powerful and Interpretable Models withRegression Networks [2.2049183478692584]
  We propose a novel architecture, Regression Networks, which combines the power of neural networks with the understandability of regression analysis. We demonstrate that the models exceed the state-of-the-art performance of interpretable models on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-07-30T03:37:00Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Fully differentiable model discovery [0.0]
  We propose an approach by combining neural network based surrogates with Sparse Bayesian Learning. Our work expands PINNs to various types of neural network architectures, and connects neural network-based surrogates to the rich field of Bayesian parameter inference.
  arXiv  Detail & Related papers  (2021-06-09T08:11:23Z)
• deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression [1.4909973741292273]
  deepregression is implemented in both R and Python, using the deep learning libraries and PyTorch, respectively. deepregression is implemented in both R and Python, using the deep learning libraries and PyTorch, respectively.
  arXiv  Detail & Related papers  (2021-04-06T17:56:31Z)
• SPINN: Sparse, Physics-based, and Interpretable Neural Networks for PDEs [0.0]
  We introduce a class of Sparse, Physics-based, and Interpretable Neural Networks (SPINN) for solving ordinary and partial differential equations. By reinterpreting a traditional meshless representation of solutions of PDEs as a special sparse deep neural network, we develop a class of sparse neural network architectures that are interpretable.
  arXiv  Detail & Related papers  (2021-02-25T17:45:50Z)
• Model-Based Deep Learning [155.063817656602]
  Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
  arXiv  Detail & Related papers  (2020-12-15T16:29:49Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.