Convolutional Motif Kernel Networks
- URL: http://arxiv.org/abs/2111.02272v3
- Date: Fri, 6 Oct 2023 12:57:45 GMT
- Title: Convolutional Motif Kernel Networks
- Authors: Jonas C. Ditz, Bernhard Reuter, Nico Pfeifer
- Abstract summary: We show that our model is able to robustly learn on small datasets and reaches state-of-the-art performance on relevant healthcare prediction tasks.
Our proposed method can be utilized on DNA and protein sequences.
- Score: 1.104960878651584
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Artificial neural networks show promising performance in detecting
correlations within data that are associated with specific outcomes. However,
the black-box nature of such models can hinder the knowledge advancement in
research fields by obscuring the decision process and preventing scientist to
fully conceptualize predicted outcomes. Furthermore, domain experts like
healthcare providers need explainable predictions to assess whether a predicted
outcome can be trusted in high stakes scenarios and to help them integrating a
model into their own routine. Therefore, interpretable models play a crucial
role for the incorporation of machine learning into high stakes scenarios like
healthcare. In this paper we introduce Convolutional Motif Kernel Networks, a
neural network architecture that involves learning a feature representation
within a subspace of the reproducing kernel Hilbert space of the position-aware
motif kernel function. The resulting model enables to directly interpret and
evaluate prediction outcomes by providing a biologically and medically
meaningful explanation without the need for additional post-hoc analysis. We
show that our model is able to robustly learn on small datasets and reaches
state-of-the-art performance on relevant healthcare prediction tasks. Our
proposed method can be utilized on DNA and protein sequences. Furthermore, we
show that the proposed method learns biologically meaningful concepts directly
from data using an end-to-end learning scheme.
Related papers
- Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network.
Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
arXiv Detail & Related papers (2024-10-31T22:54:34Z) - Seeing Unseen: Discover Novel Biomedical Concepts via
Geometry-Constrained Probabilistic Modeling [53.7117640028211]
We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues.
We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space.
A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
arXiv Detail & Related papers (2024-03-02T00:56:05Z) - Unraveling Feature Extraction Mechanisms in Neural Networks [10.13842157577026]
We propose a theoretical approach based on Neural Tangent Kernels (NTKs) to investigate such mechanisms.
We reveal how these models leverage statistical features during gradient descent and how they are integrated into final decisions.
We find that while self-attention and CNN models may exhibit limitations in learning n-grams, multiplication-based models seem to excel in this area.
arXiv Detail & Related papers (2023-10-25T04:22:40Z) - Deep networks for system identification: a Survey [56.34005280792013]
System identification learns mathematical descriptions of dynamic systems from input-output data.
Main aim of the identified model is to predict new data from previous observations.
We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
arXiv Detail & Related papers (2023-01-30T12:38:31Z) - Visual Interpretable and Explainable Deep Learning Models for Brain
Tumor MRI and COVID-19 Chest X-ray Images [0.0]
We evaluate attribution methods for illuminating how deep neural networks analyze medical images.
We attribute predictions from brain tumor MRI and COVID-19 chest X-ray datasets made by recent deep convolutional neural network models.
arXiv Detail & Related papers (2022-08-01T16:05:14Z) - Inducing Gaussian Process Networks [80.40892394020797]
We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points.
The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains.
We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
arXiv Detail & Related papers (2022-04-21T05:27:09Z) - Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs.
By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
arXiv Detail & Related papers (2022-02-01T17:11:13Z) - Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
We propose a deep co-attention network for multi-view subspace learning.
It aims to extract both the common information and the complementary information in an adversarial setting.
In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
arXiv Detail & Related papers (2021-02-15T18:46:44Z) - Interpretable Neural Networks for Panel Data Analysis in Economics [10.57079240576682]
We propose a class of interpretable neural network models that can achieve both high prediction accuracy and interpretability.
We apply the model to predicting individual's monthly employment status using high-dimensional administrative data.
We achieve an accuracy of 94.5% in the test set, which is comparable to the best performed conventional machine learning methods.
arXiv Detail & Related papers (2020-10-11T18:40:22Z) - An Investigation of Interpretability Techniques for Deep Learning in
Predictive Process Analytics [2.162419921663162]
This paper explores interpretability techniques for two of the most successful learning algorithms in medical decision-making literature: deep neural networks and random forests.
We learn models that try to predict the type of cancer of the patient, given their set of medical activity records.
We see certain distinct features used for predictions that provide useful insights about the type of cancer, along with features that do not generalize well.
arXiv Detail & Related papers (2020-02-21T09:14:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.