Optimal Invariant Bases for Atomistic Machine Learning

• URL: http://arxiv.org/abs/2503.23515v2
• Date: Thu, 03 Apr 2025 16:35:44 GMT
• Title: Optimal Invariant Bases for Atomistic Machine Learning
• Abstract summary: Many machine learning models describe the local environment of atoms, but many are incomplete or functionally dependent.<n>We employ techniques from pattern recognition literature to existing atomistic representations to produce the smallest possible set that satisfies completeness.<n>We show that this yields a new message-passing network architecture that can recognize up to 5-body patterns in each neuron.
• Score: 1.8849301976115886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The representation of atomic configurations for machine learning models has led to the development of numerous descriptors, often to describe the local environment of atoms. However, many of these representations are incomplete and/or functionally dependent. Incomplete descriptor sets are unable to represent all meaningful changes in the atomic environment. Complete constructions of atomic environment descriptors, on the other hand, often suffer from a high degree of functional dependence, where some descriptors can be written as functions of the others. These redundant descriptors do not provide additional power to discriminate between different atomic environments and increase the computational burden. By employing techniques from the pattern recognition literature to existing atomistic representations, we remove descriptors that are functions of other descriptors to produce the smallest possible set that satisfies completeness. We apply this in two ways: first we refine an existing description, the Atomistic Cluster Expansion. We show that this yields a more efficient subset of descriptors. Second, we augment an incomplete construction based on a scalar neural network, yielding a new message-passing network architecture that can recognize up to 5-body patterns in each neuron by taking advantage of an optimal set of Cartesian tensor invariants. This architecture shows strong accuracy on state-of-the-art benchmarks while retaining low computational cost. Our results not only yield improved models, but point the way to classes of invariant bases that minimize cost while maximizing expressivity for a host of applications.

Related papers

• Using large language models to probe the limits of atom-centered structural descriptors [1.7188280334580195]
  We report examples of 3D structures that are indistinguishable even if one considers clusters of up to seven neighbors.<n>Key ingredients in their construction can be traced to results that have been known for decades in different communities.
  arXiv  Detail & Related papers  (2026-07-29T14:40:59Z)
• Unlocking Prototype Potential: An Efficient Tuning Framework for Few-Shot Class-Incremental Learning [69.28860905525057]
  Few-shot class-incremental learning (FSCIL) seeks to continuously learn new classes from very limited samples.<n>We introduce an efficient prototype fine-tuning framework that evolves static centroids into dynamic, learnable components.
  arXiv  Detail & Related papers  (2026-02-05T03:50:53Z)
• Improving Set Function Approximation with Quasi-Arithmetic Neural Networks [23.73257235603082]
  We propose quasi-arithmetic neural networks (QUANNs)<n>QUANNs are universal approximators for a broad class of common set-function decompositions.<n>We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions.
  arXiv  Detail & Related papers  (2026-02-04T18:36:31Z)
• Heat Kernel Goes Topological [4.604003661048267]
  Topological neural networks have emerged as powerful successors of graph neural networks.<n>We introduce a novel topological framework that introduces a Laplacian operator on complexes (CCs)<n>Our approach captures multiscale information and enables permutation-equivariant representations, allowing easy integration into modern transformer-based architectures.
  arXiv  Detail & Related papers  (2025-07-16T16:28:10Z)
• Sparse Fine-Tuning of Transformers for Generative Tasks [25.83571722912947]
  We introduce a fine-tuning framework inspired by sparse coding, where fine-tuned features are represented as a sparse combination of basic elements.<n>The feature dictionary atoms function as fundamental building blocks of the representation, and tuning atoms allows for seamless adaptation to downstream tasks.
  arXiv  Detail & Related papers  (2025-07-14T23:03:24Z)
• Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models? [68.72260770171212]
  We propose a paradigm of Self-structured Chain of Thought (SCoT), which is composed of minimal semantic atomic steps.<n>Our method can not only generate cognitive CoT structures for various complex tasks but also mitigates the phenomenon of overthinking.<n>We conduct extensive experiments to show that the proposed AtomThink significantly improves the performance of baseline MLLMs.
  arXiv  Detail & Related papers  (2025-03-08T15:23:47Z)
• LATTE: an atomic environment descriptor based on Cartesian tensor contractions [3.2771631221674333]
  We propose a new descriptor for local atomic environments, to be used in combination with machine learning models for the construction of interatomic potentials. The Local Atomics Trainable Expansion (LATTE) allows for the efficient construction of a variable number of many-body terms with learnable parameters. We compare this new descriptor to existing ones on several systems, showing it to be competitive with very fast potentials at one end of the spectrum, and to an accuracy close to the state of the art.
  arXiv  Detail & Related papers  (2024-05-13T19:31:00Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Towards Practical Explainability with Cluster Descriptors [3.899688920770429]
  We study the problem of making the clusters more explainable by investigating the cluster descriptors. The goal is to find a representative set of tags for each cluster, referred to as the cluster descriptors, with the constraint that these descriptors are pairwise disjoint. We propose a novel explainability model that reinforces the previous models in such a way that tags that do not contribute to explainability and do not sufficiently distinguish between clusters are not added to the optimal descriptors.
  arXiv  Detail & Related papers  (2022-10-18T01:53:43Z)
• Equivariant Transduction through Invariant Alignment [71.45263447328374]
  We introduce a novel group-equivariant architecture that incorporates a group-in hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariant properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task.
  arXiv  Detail & Related papers  (2022-09-22T11:19:45Z)
• Persistent homology-based descriptor for machine-learning potential of amorphous structures [0.0]
  High-accuracy prediction of the physical properties of amorphous materials is challenging in condensed-matter physics. A promising method to achieve this is machine-learning potentials, which is an alternative to computationally demanding ab initio calculations. We propose a novel descriptor based on a persistence diagram (PD), a two-dimensional representation of persistent homology (PH)
  arXiv  Detail & Related papers  (2022-06-28T03:29:08Z)
• Scalable Interpretability via Polynomials [33.51591891812176]
  Generalized Additive Models (GAMs) have quickly become the leading choice for fully-interpretable machine learning. Unlike uninterpretable methods such as DNNs, they lack power and easy scalability, and are hence not a feasible alternative for real-world tasks. We present a new class of GAMs that use tensor decompositions of models to learn powerful, $textitfully-interpretable$ higher-order interactions.
  arXiv  Detail & Related papers  (2022-05-27T17:19:05Z)
• The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials [3.864610688022996]
  NequIP is a message passing neural network with equivariant features that showed state of the art accuracy. We construct a mathematical framework that unifies these models. We present BOTNet, a much-simplified version of NequIP, which has an interpretable architecture and maintains accuracy on benchmark datasets.
  arXiv  Detail & Related papers  (2022-05-13T13:47:12Z)
• Frame Averaging for Invariant and Equivariant Network Design [50.87023773850824]
  We introduce Frame Averaging (FA), a framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. We show that FA-based models have maximal expressive power in a broad setting. We propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs.
  arXiv  Detail & Related papers  (2021-10-07T11:05:23Z)
• Captum: A unified and generic model interpretability library for PyTorch [49.72749684393332]
  We introduce a novel, unified, open-source model interpretability library for PyTorch. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms. It can be used for both classification and non-classification models.
  arXiv  Detail & Related papers  (2020-09-16T18:57:57Z)
• Discontinuous Constituent Parsing with Pointer Networks [0.34376560669160383]
  discontinuous constituent trees are crucial for representing all grammatical phenomena of languages such as German. Recent advances in dependency parsing have shown that Pointer Networks excel in efficiently parsing syntactic relations between words in a sentence. We propose a novel neural network architecture that is able to generate the most accurate discontinuous constituent representations.
  arXiv  Detail & Related papers  (2020-02-05T15:12:03Z)
• MGCN: Descriptor Learning using Multiscale GCNs [50.14172863706108]
  We present a new non-learned feature that uses graph wavelets to decompose the Dirichlet energy on a surface. We also propose a new graph convolutional network (MGCN) to transform a non-learned feature to a more discriminative descriptor.
  arXiv  Detail & Related papers  (2020-01-28T17:25:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.