PaMM: Periodic Motif Memory for Atomistic Models with an Explicit Local-Structure Interface

• URL: http://arxiv.org/abs/2605.13297v1
• Date: Wed, 13 May 2026 10:11:27 GMT
• Title: PaMM: Periodic Motif Memory for Atomistic Models with an Explicit Local-Structure Interface
• Abstract summary: We introduce PaMM, a periodic motif memory that augments the UMA eSCN-MD edge encoder with explicit pair and triplet lookup features.<n>We evaluate PaMM in a matched UMA-S OMAT setting and focus on a narrow question: whether explicit motif memory helps at a fixed intermediate training budget.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Periodic crystals repeatedly instantiate similar local coordination motifs across translated cells and chemically related structures, but current equivariant atomistic models usually encode these patterns only implicitly in dense edge features. We introduce PaMM, a periodic motif memory that augments the UMA eSCN-MD edge encoder with explicit pair and triplet lookup features. Pair motifs are keyed by $(Z_j, Z_i, b_r)$ and triplet motifs by $(Z_j, Z_i, Z_k, b_θ)$, hashed into fixed-size tables and fused with the baseline edge representation through lightweight gate-only and affine-equipped variants. We evaluate PaMM in a matched UMA-S OMAT setting and focus on a narrow question: whether explicit motif memory helps at a fixed intermediate training budget. At the 10k-step checkpoint, both PaMM variants improve over the plain baseline; gate-only gives the best energy MAE, while the affine-equipped variant gives the best force MAE. A matched 20k follow-up keeps the same operating-point picture. Aligned controls show that the gain weakens for pair-only, triplet-only, random-bucket, and parameter-matched MLP alternatives, suggesting that the benefit is tied to structured pair/triplet organization rather than generic added capacity. A within-OMAT24 source-family evaluation also shows small but consistent gains across held-out generation families. We therefore make a focused claim: in the studied UMA-S + OMAT regime, explicit pair/ triplet motif memory is a useful inductive bias for periodic atomistic modeling. We do not claim broad cross-dataset transfer, a uniquely preferred fusion variant, or strong scientific interpretability beyond a more inspectable local-structure interface.

Related papers

• User-Centric Modeling of Transactional Sequences with Explainable State Space Models [0.0]
  We propose a hybrid approach for user-centric modeling of transactional event sequences that combines contrastive representation learning (CoLES) with State Space Models (SSMs)<n>Experiments on three public datasets demonstrate consistent improvements over standalone Mamba and CoLES with a linear classifier.
  arXiv  Detail & Related papers  (2026-07-22T14:47:25Z)
• Gaussian Mixture Attention: Linear-Time Sequence Mixing via Probabilistic Latent Routing [0.0]
  GMA replaces explicit pairwise query--key comparison with routing through $K$ learned Gaussian mixture components.<n>We formulate bidirectional and causal variants of GMA, provide an end-to-end differentiable parameterization of the Gaussian mixture components.<n>We analyze its responsibility-modulated structure, constrained non-negative low-rank affinity interpretation, and local routing stability.
  arXiv  Detail & Related papers  (2026-06-09T23:35:17Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• Geometric Factual Recall in Transformers [57.48371649045765]
  A common view casts internal weight matrices as associative memories over pairs of embeddings, requiring memorization counts that scale linearly with the number of facts.<n>We develop a theoretical and empirical account of an alternative, emphgeometric form of memorization in which learned embeddings encode structure directly.<n>In a controlled setting where a single-layer transformer must memorize random vectors from subjects to a shared attribute set, we prove that a logarithmic embedding dimension suffices.<n>We extend these results to the multi-hop setting, providing constructions with and without chain-of-thought that exhibit a provable
  arXiv  Detail & Related papers  (2026-05-12T17:22:22Z)
• Rethink MAE with Linear Time-Invariant Dynamics [0.0]
  We show that token order is a critical, exploitable dimension in frozen visual representations.<n>We propose SSMProbe, a probing framework driven by a State Space Model.
  arXiv  Detail & Related papers  (2026-04-29T15:07:00Z)
• Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for Large Language Models [9.059725329168435]
  Top-W is a geometry-aware truncation rule that uses Wasserstein distance-defined over token-embedding geometry.<n>We show that Top-W consistently outperforms prior state-of-the-art decoding approaches achieving up to 33.7% improvement.
  arXiv  Detail & Related papers  (2026-02-10T22:36:48Z)
• Cross-modal Full-mode Fine-grained Alignment for Text-to-Image Person Retrieval [54.90229711181207]
  Text-to-Image Person Retrieval (TIPR) aims to retrieve the most relevant person images based on a given text query.<n>The key challenge in TIPR lies in achieving effective alignment between textual and visual modalities.<n>We propose FMFA, a cross-modal Full-Mode Fine-grained Alignment framework.
  arXiv  Detail & Related papers  (2025-09-17T07:12:05Z)
• PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation [70.98107766265636]
  This paper takes the geometric attributes of pre-trained weights as a starting point, systematically analyzing three key components: magnitude, absolute angle, and pairwise angular structure.<n>We find that the pairwise angular structure remains stable across diverse corrupted domains and encodes domain-invariant semantic information, suggesting it should be preserved during adaptation.
  arXiv  Detail & Related papers  (2025-06-03T05:18:15Z)
• InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion [36.27704594180795]
  InfiGFusion is a structure-aware fusion framework with a novel textitGraph-on-Logits Distillation (GLD) loss.<n>We show that GLD consistently improves fusion quality and stability.<n>It shows particular strength in complex reasoning tasks, with +35.6 improvement on Multistep Arithmetic and +37.06 on Causal Judgement over SFT.
  arXiv  Detail & Related papers  (2025-05-20T03:55:35Z)
• Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods [14.96239541426242]
  We show that permutations found by WM do not significantly reduce the $L2$ distance between two models.<n>This finding shows that permutations found by WM primarily align the directions of singular vectors associated with large singular values across models.<n>This paper also analyzes activation matching (AM) in terms of singular vectors and finds that the principle of AM is likely the same as that of WM.
  arXiv  Detail & Related papers  (2024-02-06T14:53:28Z)
• Transformers as Support Vector Machines [54.642793677472724]
  We establish a formal equivalence between the optimization geometry of self-attention and a hard-margin SVM problem. We characterize the implicit bias of 1-layer transformers optimized with gradient descent. We believe these findings inspire the interpretation of transformers as a hierarchy of SVMs that separates and selects optimal tokens.
  arXiv  Detail & Related papers  (2023-08-31T17:57:50Z)
• Unsupervised Semantic Segmentation by Distilling Feature Correspondences [94.73675308961944]
  Unsupervised semantic segmentation aims to discover and localize semantically meaningful categories within image corpora without any form of annotation. We present STEGO, a novel framework that distills unsupervised features into high-quality discrete semantic labels. STEGO yields a significant improvement over the prior state of the art, on both the CocoStuff and Cityscapes challenges.
  arXiv  Detail & Related papers  (2022-03-16T06:08:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.