Selective Coupling of Decoupled Informative Regions: Masked Attention Alignment for Data-Free Quantization of Vision Transformers

• URL: http://arxiv.org/abs/2606.04373v2
• Date: Fri, 05 Jun 2026 08:17:28 GMT
• Title: Selective Coupling of Decoupled Informative Regions: Masked Attention Alignment for Data-Free Quantization of Vision Transformers
• Abstract summary: Data-Free Quantization (DFQ) addresses data security concerns by synthesizing samples, without accessing real data.<n>Previous DFQ arts for Vision Transformers (ViTs) often suffer from a distribution mismatch between synthetic samples and input distribution expected by quantized models Q.<n>We propose a novel Masked Attention Alignment approach for Data-Free Quantization of ViTs, named MaskAQ.
• Score: 56.376795859825705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-Free Quantization (DFQ) addresses data security concerns by synthesizing samples, without accessing real data. It has garnered increasing attention in the context of Vision Transformers (ViTs), owing to the superiority of the self-attention mechanism compared to classical convolutional operation. However, previous DFQ arts for ViTs often suffer from a distribution mismatch between synthetic samples and input distribution expected by quantized models Q, resulting in the suboptimal performance. In this paper, we propose a novel Masked Attention Alignment approach for Data-Free Quantization of ViTs, named MaskAQ, revealing that: 1) the semantics in the self-attention mechanism is predominantly localized to a sparse subset of patches, called informative regions; 2) the informative regions dominate the mutual information between synthetic samples and Q's outputs. To these ends, we incorporate differential entropy maximum over patch similarity of synthetic samples, to decouple informative regions from noisy background. To couple with varied Q, the informative regions are selected to align full-precision models with Q via a masked attention alignment objective, thus yielding high-quality synthetic samples. Furthermore, a periodic sample refreshing strategy comes up to endow MaskAQ with the capacity to continually adapt to the evolving state of Q throughout the training process, to preserve desirable mutual information with synthetic samples. Extensive experiments verify the merits of MaskAQ over state-of-the-art approaches across multiple backbones and downstream tasks. Our code is available at https://github.com/hfutqian/MaskAQ.

Related papers

• Towards Syn-to-Real IQA: A Novel Perspective on Reshaping Synthetic Data Distributions [74.00222571094437]
  Blind Image Quality Assessment (BIQA) has advanced significantly through deep learning, but the scarcity of large-scale labeled datasets remains a challenge.<n>We make a key observation that representations learned from synthetic datasets often exhibit a discrete and clustered pattern that hinders regression performance.<n>We introduce a novel framework SynDR-IQA, which reshapes synthetic data distribution to enhance BIQA generalization.
  arXiv  Detail & Related papers  (2026-01-01T06:11:16Z)
• S$^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation [55.35880044416441]
  We propose S$2$Q-VDiT, a post-training quantization framework for video diffusion models (V-DMs)<n>Under W4A6 quantization, S$2$Q-VDiT achieves lossless performance while delivering $3.9times$ model compression and $1.3times$ inference acceleration.
  arXiv  Detail & Related papers  (2025-08-06T02:12:29Z)
• Enhancing Generalization in Data-free Quantization via Mixup-class Prompting [8.107092196905157]
  Post-training quantization (PTQ) improves efficiency but struggles with limited calibration data, especially under privacy constraints.<n>Data-free quantization (DFQ) mitigates this by generating synthetic images using generative models such as generative adversarial networks (GANs) and text-conditioned latent diffusion models (LDMs)<n>We propose textbfmixup-class prompt, a mixup-based text prompting strategy that fuses multiple class labels at the text prompt level to generate diverse, robust synthetic data.
  arXiv  Detail & Related papers  (2025-07-29T16:00:20Z)
• Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation [70.95380821618711]
  Dichotomous Image (DIS) tasks require highly precise annotations.<n>Current generative models and techniques struggle with the issues of scene deviations, noise-induced errors, and limited training sample variability.<n>We introduce a novel approach, which provides a scalable solution for generating diverse and precise datasets.
  arXiv  Detail & Related papers  (2024-12-26T06:37:25Z)
• MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity [32.532780329341186]
  Data-free quantization (DFQ) is a technique that creates a lightweight network from its full-precision counterpart without the original training data, often through a synthetic dataset.<n>Several DFQ methods have been proposed for vision transformer (ViT) architectures, but they fail to achieve efficacy in low-bit settings.<n>We devise MimiQ, a novel DFQ method designed for ViTs that enhances inter-head attention similarity.
  arXiv  Detail & Related papers  (2024-07-29T13:57:40Z)
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation [49.827306773992376]
  Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions. Our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-12-19T15:34:52Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• ACQ: Improving Generative Data-free Quantization Via Attention Correction [2.877618279821292]
  ACQ is proposed in this paper to fix the attention of synthetic samples. Under various training settings, ACQ achieves the best quantization performance.
  arXiv  Detail & Related papers  (2023-01-18T02:13:43Z)
• Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples [8.975667614727652]
  We propose Qimera, a method that uses superposed latent embeddings to generate synthetic boundary supporting samples. The experimental results show that Qimera achieves state-of-the-art performances for various settings on data-free quantization.
  arXiv  Detail & Related papers  (2021-11-04T04:52:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.