SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models

• URL: http://arxiv.org/abs/2605.17985v1
• Date: Mon, 18 May 2026 07:40:46 GMT
• Title: SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models
• Abstract summary: We propose a new method for compressing physics foundation models (PFMs) which is a new trend in AI Science.<n>Experiments show substantial gains over existing methods across multiple models and datasets, achieving significantly higher compression ratios while maintaining accuracy, in some cases by orders of magnitude.
• Score: 8.53361303062229
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a new method for compressing physics foundation models (PFMs) which is a new trend in AI for Science. While model compression is essential for reducing memory use and accelerating inference in large foundation models, it remains under-explored for PFMs, where preserving physical fidelity is crucial. The challenge lies in the functional nature of physics data, where partial derivatives encode spatiotemporal dynamics and exhibit high sensitivity to compression. Conventional compression methods ignore this structure, often causing severe performance degradation or failure. To address this, we introduce a sensitivity-aware fidelity-enforcing compression framework that explicitly models loss-aware layer sensitivity in the output function space during compression. This provides a new route to compressing scientific foundation models while preserving accuracy and physical fidelity. Experiments show substantial gains over existing methods across multiple models and datasets, achieving significantly higher compression ratios while maintaining accuracy, in some cases by orders of magnitude. More broadly, the work potentially leads to a new subfield of efficient, deployable, and sustainable scientific foundation models in AI for Science.

Related papers

• Compressing What Matters: Neuron Importance Meets Data-Aware Low Rank Approximation for Language Model Compression [39.47298454012977]
  We propose an enhanced and computationally efficient algorithm for dynamic compression rate allocation.<n> Experimental results support the efficacy of the proposed approach which performs on par or substantially better than the previous state-of-the-art.
  arXiv  Detail & Related papers  (2026-06-30T09:16:00Z)
• Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks [10.505297793600137]
  We propose a novel method for model compression through two phases.<n>First, we utilize model compression techniques, such as pruning and quantization, to significantly reduce the model size.<n>Then, we use Mixture of Experts to route the previously compressed models to enhance performance while maintaining a balance in inference efficiency.
  arXiv  Detail & Related papers  (2026-06-22T07:11:52Z)
• Physics-Preserving Latent Compression for Zero-Shot Resolution Transfer in 3D Turbulence [23.18933468902136]
  High-resolution turbulence modeling is essential for scientific computing.<n>Existing scientific compressors reduce storage but typically operate on per-frame representations.<n>We propose Physics-Preserving Latent Compression (PPLC), a patch-local latent compressor for three-dimensional turbulence.
  arXiv  Detail & Related papers  (2026-06-19T22:02:42Z)
• D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation [78.32916244416033]
  Supervised Fine-Tuning (SFT) accelerates taskspecific large language models (LLMs) development, but the resulting proliferation of finetuned models incurs substantial memory overhead.<n>We propose DQRELO (Delta Compression via Quantization and Residual Low-Rank), a novel training- and data-free delta compression method.<n>It combines coarse-grained one-bit quantization to capture the dominant structure of the delta, followed by compensated residual low-rank approximation to recover fine-grained details from the smaller residual error.
  arXiv  Detail & Related papers  (2026-04-18T09:52:18Z)
• Big2Small: A Unifying Neural Network Framework for Model Compression [14.509737343784913]
  We construct a unifying mathematical framework for model compression grounded in measure theory.<n>We show that each model compression technique is mathematically equivalent to a neural network subject to a regularization.<n>We propose an experimentally-verified data-free model compression framework, termed textitBig2Small, which translates Implicit Neural Representations (INRs) from data domain to domain of network parameters.
  arXiv  Detail & Related papers  (2026-03-31T14:09:19Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• PhysE-Inv: A Physics-Encoded Inverse Modeling approach for Arctic Snow Depth Prediction [0.16206783799607727]
  We introduce PhysE-Inv, a novel framework that integrates a sophisticated sequential architecture, an LSTM-Decoder with Multi-head Attention and physics-guided contrastive learning, with physics-guided inference.<n> PhysE-Inv significantly improves prediction performance, reducing error by 20% while demonstrating superior physical consistency and resilience to data sparsity compared to empirical methods.<n>This approach pioneers a path for noise-tolerant, interpretable inverse modeling, with wide applicability in geospatial and cryospheric domains.
  arXiv  Detail & Related papers  (2026-01-23T00:43:51Z)
• FunDiff: Diffusion Models over Function Spaces for Physics-Informed Generative Modeling [3.6766942024793496]
  We introduce FunDiff, a novel framework for generative modeling in function spaces.<n>FunDiff combines a latent diffusion process with a function autoencoder architecture to handle input functions.<n>We demonstrate the practical effectiveness of FunDiff across diverse applications in fluid dynamics and solid mechanics.
  arXiv  Detail & Related papers  (2025-06-09T16:19:59Z)
• Shifting AI Efficiency From Model-Centric to Data-Centric Compression [67.45087283924732]
  We argue that the focus of research for AI is shifting from model-centric compression to data-centric compression.<n>Data-centric compression improves AI efficiency by directly compressing the volume of data processed during model training or inference.<n>Our work aims to provide a novel perspective on AI efficiency, synthesize existing efforts, and catalyze innovation to address the challenges posed by ever-increasing context lengths.
  arXiv  Detail & Related papers  (2025-05-25T13:51:17Z)
• Pathology Image Compression with Pre-trained Autoencoders [52.208181380986524]
  Whole Slide Images in digital histopathology pose significant storage, transmission, and computational efficiency challenges.<n>Standard compression methods, such as JPEG, reduce file sizes but fail to preserve fine-grained phenotypic details critical for downstream tasks.<n>In this work, we repurpose autoencoders (AEs) designed for Latent Diffusion Models as an efficient learned compression framework for pathology images.
  arXiv  Detail & Related papers  (2025-03-14T17:01:17Z)
• Foundation Model for Lossy Compression of Spatiotemporal Scientific Data [11.494915987840876]
  We present a foundation model (FM) for lossy scientific data compression.<n>We combine a variational autoencoder (E) with a hyper-prior structure and a super-resolution (SR) module.
  arXiv  Detail & Related papers  (2024-12-22T22:57:08Z)
• UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspective [85.08718140718707]
  UNComp is an uncertainty-aware framework that uncovers sparsity patterns that can be used for adaptive compression.<n>By focusing on uncertainty to analyze the sparsity pattern in detail, UNComp reduces the KV cache size to 4.74% of the original, achieves a 6% prefill speedup, and improves throughput by 6.4x.
  arXiv  Detail & Related papers  (2024-10-04T02:32:36Z)
• A Survey on Transformer Compression [84.18094368700379]
  Transformer plays a vital role in the realms of natural language processing (NLP) and computer vision (CV) Model compression methods reduce the memory and computational cost of Transformer. This survey provides a comprehensive review of recent compression methods, with a specific focus on their application to Transformer-based models.
  arXiv  Detail & Related papers  (2024-02-05T12:16:28Z)
• What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression [68.82486784654817]
  We study two popular model compression techniques including knowledge distillation and pruning. We show that compressed models are significantly less robust than their PLM counterparts on adversarial test sets. We develop a regularization strategy for model compression based on sample uncertainty.
  arXiv  Detail & Related papers  (2021-10-16T00:20:04Z)
• Robustness in Compressed Neural Networks for Object Detection [2.9823962001574182]
  The sensitivity of compressed models to different distortion types is nuanced. Some of the corruptions are heavily impacted by the compression methods. Data augmentation was confirmed to positively affect models' robustness.
  arXiv  Detail & Related papers  (2021-02-10T15:52:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.