A mechanistically interpretable neural network for regulatory genomics

• URL: http://arxiv.org/abs/2410.06211v1
• Date: Tue, 8 Oct 2024 17:15:26 GMT
• Title: A mechanistically interpretable neural network for regulatory genomics
• Abstract summary: We show that our architecture excels in de novo motif discovery and motif instance calling, is robust to variable sequence contexts, and enables fully interpretable generation of novel functional sequences.
• Score: 0.6224769485481241
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Deep neural networks excel in mapping genomic DNA sequences to associated readouts (e.g., protein-DNA binding). Beyond prediction, the goal of these networks is to reveal to scientists the underlying motifs (and their syntax) which drive genome regulation. Traditional methods that extract motifs from convolutional filters suffer from the uninterpretable dispersion of information across filters and layers. Other methods which rely on importance scores can be unstable and unreliable. Instead, we designed a novel mechanistically interpretable architecture for regulatory genomics, where motifs and their syntax are directly encoded and readable from the learned weights and activations. We provide theoretical and empirical evidence of our architecture's full expressivity, while still being highly interpretable. Through several experiments, we show that our architecture excels in de novo motif discovery and motif instance calling, is robust to variable sequence contexts, and enables fully interpretable generation of novel functional sequences.

Related papers

• SeqMaestro: From nucleotide sequences to biological hypotheses through interpretable machine learning [35.97164357682394]
  SeqMaestro is a machine learning framework that proposes hypotheses from nucleotide sequences using interpretable models.<n>Our solution is centered around a two-layer interface that connects nucleotide sequences with the broader ecosystem of interpretable machine learning.
  arXiv  Detail & Related papers  (2026-09-14T01:04:09Z)
• Interpretable DNA Sequence Classification via Dynamic Feature Generation in Decision Trees [48.331026745746776]
  DEFT is a novel framework that adaptively generates high-level sequence features during tree construction.<n>We show that DEFT discovers human-interpretable and highly predictive sequence features across a diverse range of genomic tasks.
  arXiv  Detail & Related papers  (2026-04-13T20:58:01Z)
• Unveiling and Steering Connectome Organization with Interpretable Latent Variables [11.702970031377307]
  We propose a framework that combines subgraph extraction from the Drosophila connectome, FlyWire, with a generative model to derive interpretable low-dimensional representations of neural circuitry.<n>This research offers a novel tool for understanding brain architecture and a potential avenue for designing bio-inspired artificial neural networks.
  arXiv  Detail & Related papers  (2025-05-19T11:54:40Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters. Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks. It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Improving Neuron-level Interpretability with White-box Language Models [11.898535906016907]
  We introduce a white-box transformer-like architecture named Coding RAte TransformEr (CRATE) Our comprehensive experiments showcase significant improvements (up to 103% relative improvement) in neuron-level interpretability. CRATE's increased interpretability comes from its enhanced ability to consistently and distinctively activate on relevant tokens.
  arXiv  Detail & Related papers  (2024-10-21T19:12:33Z)
• Semantically Rich Local Dataset Generation for Explainable AI in Genomics [0.716879432974126]
  Black box deep learning models trained on genomic sequences excel at predicting the outcomes of different gene regulatory mechanisms. We propose using Genetic Programming to generate datasets by evolving perturbations in sequences that contribute to their semantic diversity.
  arXiv  Detail & Related papers  (2024-07-03T10:31:30Z)
• VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling [60.91599380893732]
  VQDNA is a general-purpose framework that renovates genome tokenization from the perspective of genome vocabulary learning. By leveraging vector-quantized codebooks as learnable vocabulary, VQDNA can adaptively tokenize genomes into pattern-aware embeddings.
  arXiv  Detail & Related papers  (2024-05-13T20:15:03Z)
• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• Understanding polysemanticity in neural networks through coding theory [0.8702432681310401]
  We propose a novel practical approach to network interpretability and theoretical insights into polysemanticity and the density of codes. We show how random projections can reveal whether a network exhibits a smooth or non-differentiable code and hence how interpretable the code is. Our approach advances the pursuit of interpretability in neural networks, providing insights into their underlying structure and suggesting new avenues for circuit-level interpretability.
  arXiv  Detail & Related papers  (2024-01-31T16:31:54Z)
• On the Expressiveness and Generalization of Hypergraph Neural Networks [77.65788763444877]
  This extended abstract describes a framework for analyzing the expressiveness, learning, and (structural) generalization of hypergraph neural networks (HyperGNNs) Specifically, we focus on how HyperGNNs can learn from finite datasets and generalize structurally to graph reasoning problems of arbitrary input sizes.
  arXiv  Detail & Related papers  (2023-03-09T18:42:18Z)
• Neural-Symbolic Recursive Machine for Systematic Generalization [113.22455566135757]
  We introduce the Neural-Symbolic Recursive Machine (NSR), whose core is a Grounded Symbol System (GSS) NSR integrates neural perception, syntactic parsing, and semantic reasoning. We evaluate NSR's efficacy across four challenging benchmarks designed to probe systematic generalization capabilities.
  arXiv  Detail & Related papers  (2022-10-04T13:27:38Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Convolutional Motif Kernel Networks [1.104960878651584]
  We show that our model is able to robustly learn on small datasets and reaches state-of-the-art performance on relevant healthcare prediction tasks. Our proposed method can be utilized on DNA and protein sequences.
  arXiv  Detail & Related papers  (2021-11-03T15:06:09Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.