Crystalite: A Lightweight Transformer for Efficient Crystal Modeling

• URL: http://arxiv.org/abs/2604.02270v1
• Date: Thu, 02 Apr 2026 17:03:42 GMT
• Title: Crystalite: A Lightweight Transformer for Efficient Crystal Modeling
• Abstract summary: Crystalite is a lightweight diffusion Transformer built around two simple inductive biases.<n>GEM injects periodic minimum-image pair geometry directly into attention through additive geometric biases.<n>Crystalite achieves state-of-the-art results on crystal structure prediction benchmarks, and de novo generation performance.
• Score: 6.443928036308759
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative models for crystalline materials often rely on equivariant graph neural networks, which capture geometric structure well but are costly to train and slow to sample. We present Crystalite, a lightweight diffusion Transformer for crystal modeling built around two simple inductive biases. The first is Subatomic Tokenization, a compact chemically structured atom representation that replaces high-dimensional one-hot encodings and is better suited to continuous diffusion. The second is the Geometry Enhancement Module (GEM), which injects periodic minimum-image pair geometry directly into attention through additive geometric biases. Together, these components preserve the simplicity and efficiency of a standard Transformer while making it better matched to the structure of crystalline materials. Crystalite achieves state-of-the-art results on crystal structure prediction benchmarks, and de novo generation performance, attaining the best S.U.N. discovery score among the evaluated baselines while sampling substantially faster than geometry-heavy alternatives.

Related papers

• CrystalDiT: A Diffusion Transformer for Crystal Generation [52.45780803467369]
  We present CrystalDiT, a diffusion transformer for crystal structure generation that achieves state-of-the-art performance.<n>CrystalDiT employs a unified transformer that imposes a powerful inductive bias: treating lattice and atomic properties as a single, interdependent system.
  arXiv  Detail & Related papers  (2025-08-13T12:53:32Z)
• Invariant Tokenization of Crystalline Materials for Language Model Enabled Generation [82.91073155506277]
  Key step is to convert 3D crystal structures into 1D sequences to be processed by language models (LMs)<n>Mat2Seq converts 3D crystal structures into 1D sequences and ensures that different mathematical descriptions of the same crystal are represented in a single unique sequence.<n> Experimental results show that, with language models, Mat2Seq achieves promising performance in crystal structure generation as compared with prior methods.
  arXiv  Detail & Related papers  (2025-02-28T20:02:53Z)
• CrystalGRW: Generative Modeling of Crystal Structures with Targeted Properties via Geodesic Random Walks [1.2141052067494946]
  We introduce CrystalGRW, a diffusion-based generative model that can predict stable phases validated by density functional theory.<n>CrystalGRW demonstrates the ability to generate realistic crystal structures that are close to their ground states with accuracy comparable to existing models.<n>These features help accelerate materials discovery and inverse design by offering stable, symmetry-consistent crystal candidates for experimental validation.
  arXiv  Detail & Related papers  (2025-01-15T18:26:35Z)
• Space Group Informed Transformer for Crystalline Materials Generation [2.1837453116459824]
  We introduce CrystalFormer, a transformer-based autoregressive model for space group-controlled generation of crystalline materials.<n>By explicitly incorporating space group symmetry, CrystalFormer greatly reduces the effective complexity of crystal space.<n>CrystalFormer is a promising architecture to be the model of the entire crystalline materials space, heralding a new era in materials discovery and design.
  arXiv  Detail & Related papers  (2024-03-23T06:01:45Z)
• Complete and Efficient Graph Transformers for Crystal Material Property Prediction [53.32754046881189]
  Crystal structures are characterized by atomic bases within a primitive unit cell that repeats along a regular lattice throughout 3D space. We introduce a novel approach that utilizes the periodic patterns of unit cells to establish the lattice-based representation for each atom. We propose ComFormer, a SE(3) transformer designed specifically for crystalline materials.
  arXiv  Detail & Related papers  (2024-03-18T15:06:37Z)
• Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding [10.170537065646323]
  Predicting physical properties of materials from their crystal structures is a fundamental problem in materials science. We show that crystal structures are infinitely repeating, periodic arrangements of atoms, whose fully connected attention results in infinitely connected attention. We propose a simple yet effective Transformer-based encoder architecture for crystal structures called Crystalformer.
  arXiv  Detail & Related papers  (2024-03-18T11:37:42Z)
• Scalable Diffusion for Materials Generation [99.71001883652211]
  We develop a unified crystal representation that can represent any crystal structure (UniMat) UniMat can generate high fidelity crystal structures from larger and more complex chemical systems. We propose additional metrics for evaluating generative models of materials.
  arXiv  Detail & Related papers  (2023-10-18T15:49:39Z)
• Latent Conservative Objective Models for Data-Driven Crystal Structure Prediction [62.36797874900395]
  In computational chemistry, crystal structure prediction is an optimization problem. One approach to tackle this problem involves building simulators based on density functional theory (DFT) followed by running search in simulation. We show that our approach, dubbed LCOMs (latent conservative objective models), performs comparably to the best current approaches in terms of success rate of structure prediction.
  arXiv  Detail & Related papers  (2023-10-16T04:35:44Z)
• Data-Driven Score-Based Models for Generating Stable Structures with Adaptive Crystal Cells [1.515687944002438]
  This work aims at the generation of new crystal structures with desired properties, such as chemical stability and specified chemical composition. The novelty of the presented approach resides in the fact that the lattice of the crystal cell is not fixed. A multigraph crystal representation is introduced that respects symmetry constraints, yielding computational advantages.
  arXiv  Detail & Related papers  (2023-10-16T02:53:24Z)
• Crystal-GFN: sampling crystals with desirable properties and constraints [103.79058968784163]
  We introduce Crystal-GFN, a generative model of crystal structures that sequentially samples structural properties of crystalline materials. In this paper, we use as objective the formation energy per atom of a crystal structure predicted by a new proxy machine learning model trained on MatBench. The results demonstrate that Crystal-GFN is able to sample highly diverse crystals with low (median -3.1 eV/atom) predicted formation energy.
  arXiv  Detail & Related papers  (2023-10-07T21:36:55Z)
• Shotgun crystal structure prediction using machine-learned formation energies [3.2563787689949133]
  Crystal structures of assembled atoms can be predicted by finding the global or local minima of the energy surface within a broad space of atomic configurations. Here, we present significant progress toward solving the crystal structure prediction problem. We performed noniterative, single-shot screening using a large library of virtually created crystal structures with a machine-learning energy predictor.
  arXiv  Detail & Related papers  (2023-05-03T14:46:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.