On the Generalization in Topology Optimization via Sensitivity-Conditioned Bernoulli Flow Matching

• URL: http://arxiv.org/abs/2606.02179v1
• Date: Mon, 01 Jun 2026 12:36:41 GMT
• Title: On the Generalization in Topology Optimization via Sensitivity-Conditioned Bernoulli Flow Matching
• Abstract summary: Surrogate models for topology optimization (TO) exhibit highly variable out-of-distribution (OOD) generalization under changing loads or boundary conditions.<n>We hypothesize that OOD performance is governed by how much information the conditioning signal preserves about the adjoint sensitivity that drives classical TO.<n>We show that conditioning on sensitivities yields state-of-the-art OOD performance, while increasingly distant physical fields degrade toward raw parameter conditioning.
• Score: 18.677394555631086
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Surrogate models for topology optimization (TO) exhibit highly variable out-of-distribution (OOD) generalization under distribution shifts such as changing loads or boundary conditions, yet the source of this variability remains unclear. We hypothesize that OOD performance is governed by how much information the conditioning signal preserves about the adjoint sensitivity (reduced gradient) that drives classical TO. Modeling the TO pipeline as a causal Markov chain, the Data Processing Inequality establishes that, under this abstraction, the sensitivity field is an information-theoretically optimal conditioning signal for topology prediction. However, computing exact adjoint sensitivities can be expensive or unavailable in practice; we observe that certain physical fields can approximate sensitivities through monotone transformations. To formalize this, we introduce \textbf{pseudo-sensitivities} to characterize which fields enable generalization versus those that are information-poor. We then show that a sensitivity-conditioned Bernoulli flow-matching generator empirically confirms these predictions: conditioning on sensitivities yields state-of-the-art OOD performance, while increasingly distant physical fields degrade toward raw parameter conditioning. Results hold across structural TO benchmarks under load shifts and our new CFD-TO dataset under boundary-condition shifts such as multi-outlet configurations. Code and datasets are available at https://tum-pbs.github.io/topotransformer/ .

Related papers

• Sensitivity-Constrained Neural Operators for Data-Efficient Forward and Inverse Modeling of Partial Differential Equation Systems [7.72071408793585]
  We study sensitivity-constrained neural operators (SC-NOs)<n>SC-NOs augment standard training with solver-derived Jacobian supervision.<n>We evaluate SC-NOs on advection--diffusion and RANS--Spalart--Allmaras benchmarks, input-dimensionality scaling tests, long-horizon autoregressive rollout, and a shallow-water Tohoku tsunami source-inversion case.
  arXiv  Detail & Related papers  (2026-08-30T16:37:25Z)
• MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation [65.068801413044]
  Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams.<n>We introduce MoASE, a mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture.
  arXiv  Detail & Related papers  (2026-05-18T01:52:12Z)
• Soft Graph Diffusion Transformer for MIMO Detection [26.149951750258026]
  We propose the Soft Graph Diffusion Transformer (SGDiT), which reformulates detection as a noise-level-conditioned denoising process.<n>An adaptive layer normalization (AdaLN)-conditioned soft graph transformer is employed to parameterize the denoising dynamics.<n>We show that SGDiT achieves competitive bit error rate (BER) performance compared with representative baselines.
  arXiv  Detail & Related papers  (2026-05-01T06:36:14Z)
• Spatially-informed transformers: Injecting geostatistical covariance biases into self-attention for spatio-temporal forecasting [0.0]
  We propose a hybrid architecture that injects a geostatistic inductive bias directly into the decomposing self-attention mechanism via a learnable costatistics kernel.<n>We demonstrate the phenomenon of Deep Variography'', where the network successfully recovers the true spatial parameters of the underlying process end-to-end via backpropagation.
  arXiv  Detail & Related papers  (2025-12-19T15:32:24Z)
• Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding [5.2482659629416535]
  We characterize when tokens move closer to or farther from one another over time, depending on the model parameters.<n>We investigate how different forms of positional encoding -- specifically absolute and rotary -- affect these dynamical regimes.<n>Motivated by these insights, we propose simple refinements to Transformer architectures that mitigate convergence behavior in models with absolute or rotary positional encoding.
  arXiv  Detail & Related papers  (2025-11-25T19:39:57Z)
• Transformers Learn Faster with Semantic Focus [57.97235825738412]
  We study sparse transformers in terms of learnability and generalization.<n>We find that input-dependent sparse attention models appear to converge faster and generalize better than standard attention models.
  arXiv  Detail & Related papers  (2025-06-17T01:19:28Z)
• Neural Conditional Transport Maps [0.0]
  We introduce a conditioning mechanism capable of processing both categorical and continuous conditioning variables simultaneously.<n>At the core of our method lies a hypernetwork that generates transport layer parameters based on these inputs, creating adaptive mappings.<n>This work advances the state-of-the-art in conditional optimal transport, enabling broader application of optimal transport principles to complex, high-dimensional domains.
  arXiv  Detail & Related papers  (2025-05-21T17:59:02Z)
• Partial Transportability for Domain Generalization [56.37032680901525]
  Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution.<n>Our contribution is to provide the first general estimation technique for transportability problems.<n>We propose a gradient-based optimization scheme for making scalable inferences in practice.
  arXiv  Detail & Related papers  (2025-03-30T22:06:37Z)
• On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent [51.50999191584981]
  Sign Gradient Descent (SignGD) serves as an effective surrogate for Adam.<n>We study how SignGD optimize a two-layer transformer on a noisy dataset.<n>We find that the poor generalization of SignGD is not solely due to data noise, suggesting that both SignGD and Adam requires high-quality data for real-world tasks.
  arXiv  Detail & Related papers  (2024-10-07T09:36:43Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• Positive-definite parametrization of mixed quantum states with deep neural networks [0.0]
  We show how to embed an autoregressive structure in the GHDO to allow direct sampling of the probability distribution. We benchmark this architecture by the steady state of the dissipative transverse-field Ising model.
  arXiv  Detail & Related papers  (2022-06-27T17:51:38Z)
• Disentangling Generative Factors of Physical Fields Using Variational Autoencoders [0.0]
  This work explores the use of variational autoencoders (VAEs) for non-linear dimension reduction. A disentangled decomposition is interpretable and can be transferred to a variety of tasks including generative modeling.
  arXiv  Detail & Related papers  (2021-09-15T16:02:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.