Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding

• URL: http://arxiv.org/abs/2403.11686v1
• Date: Mon, 18 Mar 2024 11:37:42 GMT
• Title: Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding
• Authors: Tatsunori Taniai, Ryo Igarashi, Yuta Suzuki, Naoya Chiba, Kotaro Saito, Yoshitaka Ushiku, Kanta Ono,
• Abstract summary: Predicting physical properties of materials from their crystal structures is a fundamental problem in materials science. We show that crystal structures are infinitely repeating, periodic arrangements of atoms, whose fully connected attention results in infinitely connected attention. We propose a simple yet effective Transformer-based encoder architecture for crystal structures called Crystalformer.
• Score: 10.170537065646323
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Predicting physical properties of materials from their crystal structures is a fundamental problem in materials science. In peripheral areas such as the prediction of molecular properties, fully connected attention networks have been shown to be successful. However, unlike these finite atom arrangements, crystal structures are infinitely repeating, periodic arrangements of atoms, whose fully connected attention results in infinitely connected attention. In this work, we show that this infinitely connected attention can lead to a computationally tractable formulation, interpreted as neural potential summation, that performs infinite interatomic potential summations in a deeply learned feature space. We then propose a simple yet effective Transformer-based encoder architecture for crystal structures called Crystalformer. Compared to an existing Transformer-based model, the proposed model requires only 29.4% of the number of parameters, with minimal modifications to the original Transformer architecture. Despite the architectural simplicity, the proposed method outperforms state-of-the-art methods for various property regression tasks on the Materials Project and JARVIS-DFT datasets.

Related papers

• Unleashing the power of novel conditional generative approaches for new materials discovery [3.972733741872872]
  We propose two generative approaches to the problem of crystal structure design. One is conditional structure modification, using the energy difference between the most energetically favorable structure and all its less stable polymorphs. The other is conditional structure generation, using the energy difference between the most energetically favorable structure and all its less stable polymorphs.
  arXiv  Detail & Related papers  (2024-11-05T14:58:31Z)
• CrysAtom: Distributed Representation of Atoms for Crystal Property Prediction [0.0]
  In material science literature, it is well-known that crystalline materials exhibit topological structures. In this paper, we propose an unsupervised framework namely, CrysAtom, using untagged crystal data to generate dense vector representation of atoms.
  arXiv  Detail & Related papers  (2024-09-07T06:58:55Z)
• Generative Inverse Design of Crystal Structures via Diffusion Models with Transformers [1.2289361708127877]
  New inorganic materials with promising properties pose a critical challenge, both scientifically and for industrial applications. Discovery of new inorganic materials with promising properties poses a critical challenge, both scientifically and for industrial applications. In this study, we explore a new type of diffusion model for the generative inverse design of crystal structures, with a backbone based on a Transformer architecture.
  arXiv  Detail & Related papers  (2024-06-13T16:03:15Z)
• Complete and Efficient Graph Transformers for Crystal Material Property Prediction [53.32754046881189]
  Crystal structures are characterized by atomic bases within a primitive unit cell that repeats along a regular lattice throughout 3D space. We introduce a novel approach that utilizes the periodic patterns of unit cells to establish the lattice-based representation for each atom. We propose ComFormer, a SE(3) transformer designed specifically for crystalline materials.
  arXiv  Detail & Related papers  (2024-03-18T15:06:37Z)
• Scalable Diffusion for Materials Generation [99.71001883652211]
  We develop a unified crystal representation that can represent any crystal structure (UniMat) UniMat can generate high fidelity crystal structures from larger and more complex chemical systems. We propose additional metrics for evaluating generative models of materials.
  arXiv  Detail & Related papers  (2023-10-18T15:49:39Z)
• Latent Conservative Objective Models for Data-Driven Crystal Structure Prediction [62.36797874900395]
  In computational chemistry, crystal structure prediction is an optimization problem. One approach to tackle this problem involves building simulators based on density functional theory (DFT) followed by running search in simulation. We show that our approach, dubbed LCOMs (latent conservative objective models), performs comparably to the best current approaches in terms of success rate of structure prediction.
  arXiv  Detail & Related papers  (2023-10-16T04:35:44Z)
• Crystal-GFN: sampling crystals with desirable properties and constraints [103.79058968784163]
  We introduce Crystal-GFN, a generative model of crystal structures that sequentially samples structural properties of crystalline materials. In this paper, we use as objective the formation energy per atom of a crystal structure predicted by a new proxy machine learning model trained on MatBench. The results demonstrate that Crystal-GFN is able to sample highly diverse crystals with low (median -3.1 eV/atom) predicted formation energy.
  arXiv  Detail & Related papers  (2023-10-07T21:36:55Z)
• Molecular Geometry-aware Transformer for accurate 3D Atomic System modeling [51.83761266429285]
  We propose a novel Transformer architecture that takes nodes (atoms) and edges (bonds and nonbonding atom pairs) as inputs and models the interactions among them. Moleformer achieves state-of-the-art on the initial state to relaxed energy prediction of OC20 and is very competitive in QM9 on predicting quantum chemical properties.
  arXiv  Detail & Related papers  (2023-02-02T03:49:57Z)
• Neural Structure Fields with Application to Crystal Structure Autoencoders [10.680545976155173]
  We propose neural structure fields (NeSF) as an accurate and practical approach for representing crystal structures using neural networks. NeSF overcomes the tradeoff between spatial resolution and computational complexity and can represent any crystal structure. We propose an autoencoder of crystal structures that can recover various crystal structures, such as those of perovskite structure materials and cuprate superconductors.
  arXiv  Detail & Related papers  (2022-12-08T16:41:41Z)
• Atomic structure generation from reconstructing structural fingerprints [1.2128971613239876]
  We present an end-to-end structure generation approach using atom-centered symmetry functions as the representation and conditional variational autoencoders as the generative model. We are able to successfully generate novel and valid atomic structures of sub-nanometer Pt nanoparticles as a proof of concept.
  arXiv  Detail & Related papers  (2022-07-27T00:42:59Z)
• Geometric Transformer for End-to-End Molecule Properties Prediction [92.28929858529679]
  We introduce a Transformer-based architecture for molecule property prediction, which is able to capture the geometry of the molecule. We modify the classical positional encoder by an initial encoding of the molecule geometry, as well as a learned gated self-attention mechanism.
  arXiv  Detail & Related papers  (2021-10-26T14:14:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.