Sparse generative modeling via parameter-reduction of Boltzmann machines: application to protein-sequence families

• URL: http://arxiv.org/abs/2011.11259v3
• Date: Fri, 30 Jul 2021 08:27:01 GMT
• Title: Sparse generative modeling via parameter-reduction of Boltzmann machines: application to protein-sequence families
• Authors: Pierre Barrat-Charlaix, Anna Paola Muntoni, Kai Shimagaki, Martin Weigt, Francesco Zamponi
• Abstract summary: Boltzmann machines (BM) are widely used as generative models. We introduce a general parameter-reduction procedure for BMs. For several protein families, our procedure allows one to remove more than $90%$ of the PM couplings.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Boltzmann machines (BM) are widely used as generative models. For example, pairwise Potts models (PM), which are instances of the BM class, provide accurate statistical models of families of evolutionarily related protein sequences. Their parameters are the local fields, which describe site-specific patterns of amino-acid conservation, and the two-site couplings, which mirror the coevolution between pairs of sites. This coevolution reflects structural and functional constraints acting on protein sequences during evolution. The most conservative choice to describe the coevolution signal is to include all possible two-site couplings into the PM. This choice, typical of what is known as Direct Coupling Analysis, has been successful for predicting residue contacts in the three-dimensional structure, mutational effects, and in generating new functional sequences. However, the resulting PM suffers from important over-fitting effects: many couplings are small, noisy and hardly interpretable; the PM is close to a critical point, meaning that it is highly sensitive to small parameter perturbations. In this work, we introduce a general parameter-reduction procedure for BMs, via a controlled iterative decimation of the less statistically significant couplings, identified by an information-based criterion that selects either weak or statistically unsupported couplings. For several protein families, our procedure allows one to remove more than $90\%$ of the PM couplings, while preserving the predictive and generative properties of the original dense PM, and the resulting model is far away from criticality, hence more robust to noise.

Related papers

• JanusDDG: A Thermodynamics-Compliant Model for Sequence-Based Protein Stability via Two-Fronts Multi-Head Attention [0.0]
  understanding how residue variations affect protein stability is crucial for designing functional proteins. Recent advances in protein language models (PLMs) have revolutionized computational protein analysis. We introduce JanusDDG, a deep learning framework that leverages PLM-derived embeddings and a bidirectional cross-attention transformer architecture.
  arXiv  Detail & Related papers  (2025-04-04T09:02:32Z)
• Generative Intervention Models for Causal Perturbation Modeling [80.72074987374141]
  In many applications, it is a priori unknown which mechanisms of a system are modified by an external perturbation. We propose a generative intervention model (GIM) that learns to map these perturbation features to distributions over atomic interventions.
  arXiv  Detail & Related papers  (2024-11-21T10:37:57Z)
• Multiview Random Vector Functional Link Network for Predicting DNA-Binding Proteins [0.0]
  We propose a novel framework termed a multiview random vector functional link (MvRVFL) network, which fuses neural network architecture with multiview learning. The proposed MvRVFL model combines the benefits of late and early fusion, allowing for distinct regularization parameters across different views. The performance of the proposed MvRVFL model on the DBP dataset surpasses that of baseline models, demonstrating its superior effectiveness.
  arXiv  Detail & Related papers  (2024-09-04T10:14:17Z)
• Mutagenesis screen to map the functions of parameters of Large Language Models [10.19684167876245]
  We used a mutagenesis screen approach inspired by the methods used in biological studies to investigate Llama2-7b and Zephyr. Mutations that produced phenotypes, especially those with severe outcomes, tended to cluster along axes. In Zephyr, certain mutations consistently resulted in poetic or conversational rather than descriptive outputs.
  arXiv  Detail & Related papers  (2024-08-21T10:10:08Z)
• Learning to Predict Mutation Effects of Protein-Protein Interactions by Microenvironment-aware Hierarchical Prompt Learning [78.38442423223832]
  We develop a novel codebook pre-training task, namely masked microenvironment modeling. We demonstrate superior performance and training efficiency over state-of-the-art pre-training-based methods in mutation effect prediction.
  arXiv  Detail & Related papers  (2024-05-16T03:53:21Z)
• Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels [14.186776881154127]
  This paper focuses on empirical risk minimization in two settings, namely, random features and the neural tangent kernel (NTK) We prove that, for random features, the model is not robust for any degree of over- parameterization, even when the necessary condition coming from the universal law of robustness is satisfied. Our results are corroborated by numerical evidence on both synthetic and standard prototypical datasets.
  arXiv  Detail & Related papers  (2023-02-03T09:58:31Z)
• Noise-resilient Edge Modes on a Chain of Superconducting Qubits [103.93329374521808]
  Inherent symmetry of a quantum system may protect its otherwise fragile states. We implement the one-dimensional kicked Ising model which exhibits non-local Majorana edge modes (MEMs) with $mathbbZ$ parity symmetry. MEMs are found to be resilient against certain symmetry-breaking noise owing to a prethermalization mechanism.
  arXiv  Detail & Related papers  (2022-04-24T22:34:15Z)
• Learning Generalized Gumbel-max Causal Mechanisms [31.64007831043909]
  We argue for choosing a causal mechanism that is best under a quantitative criteria such as minimizing variance when estimating counterfactual treatment effects. We show that they can be trained to minimize counterfactual effect variance and other losses on a distribution of queries of interest.
  arXiv  Detail & Related papers  (2021-11-11T22:02:20Z)
• Understanding Interlocking Dynamics of Cooperative Rationalization [90.6863969334526]
  Selective rationalization explains the prediction of complex neural networks by finding a small subset of the input that is sufficient to predict the neural model output. We reveal a major problem with such cooperative rationalization paradigm -- model interlocking. We propose a new rationalization framework, called A2R, which introduces a third component into the architecture, a predictor driven by soft attention as opposed to selection.
  arXiv  Detail & Related papers  (2021-10-26T17:39:18Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models [53.17320541056843]
  We propose a fully-differentiable approach for protein structure optimization, guided by a data-driven generative network. Our EBM-Fold approach can efficiently produce high-quality decoys, compared against traditional Rosetta-based structure optimization routines.
  arXiv  Detail & Related papers  (2021-05-11T03:40:29Z)
• Generative Capacity of Probabilistic Protein Sequence Models [0.0]
  Potts models and variational autoencoders (VAEs) have recently gained popularity as generative protein sequence models (GPSMs) It is currently unclear whether GPSMs can faithfully reproduce the complex multi-residue mutation patterns observed in natural sequences arising due to epistasis. We develop a set of sequence statistics to assess the "generative capacity" of three GPSMs of recent interest.
  arXiv  Detail & Related papers  (2020-12-03T21:59:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.