BiGain: Unified Token Compression for Joint Generation and Classification

• URL: http://arxiv.org/abs/2603.12240v1
• Date: Thu, 12 Mar 2026 17:55:53 GMT
• Title: BiGain: Unified Token Compression for Joint Generation and Classification
• Abstract summary: BiGain is a training-free, plug-and-play framework that preserves generation quality while improving classification in accelerated diffusion models.<n>Our key insight is frequency separation, which disentangles fine detail from global semantics, enabling compression that respects both generative fidelity and discriminative utility.<n>Our analyses indicate that balanced spectral retention, preserving high-frequency detail and low/mid-frequency semantics, is a reliable design rule for token compression in diffusion models.
• Score: 47.040577759493004
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Acceleration methods for diffusion models (e.g., token merging or downsampling) typically optimize synthesis quality under reduced compute, yet often ignore discriminative capacity. We revisit token compression with a joint objective and present BiGain, a training-free, plug-and-play framework that preserves generation quality while improving classification in accelerated diffusion models. Our key insight is frequency separation: mapping feature-space signals into a frequency-aware representation disentangles fine detail from global semantics, enabling compression that respects both generative fidelity and discriminative utility. BiGain reflects this principle with two frequency-aware operators: (1) Laplacian-gated token merging, which encourages merges among spectrally smooth tokens while discouraging merges of high-contrast tokens, thereby retaining edges and textures; and (2) Interpolate-Extrapolate KV Downsampling, which downsamples keys/values via a controllable interextrapolation between nearest and average pooling while keeping queries intact, thereby conserving attention precision. Across DiT- and U-Net-based backbones and ImageNet-1K, ImageNet-100, Oxford-IIIT Pets, and COCO-2017, our operators consistently improve the speed-accuracy trade-off for diffusion-based classification, while maintaining or enhancing generation quality under comparable acceleration. For instance, on ImageNet-1K, with 70% token merging on Stable Diffusion 2.0, BiGain increases classification accuracy by 7.15% while improving FID by 0.34 (1.85%). Our analyses indicate that balanced spectral retention, preserving high-frequency detail and low/mid-frequency semantics, is a reliable design rule for token compression in diffusion models. To our knowledge, BiGain is the first framework to jointly study and advance both generation and classification under accelerated diffusion, supporting lower-cost deployment.

Related papers

• SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation [52.62632888060867]
  Visible-to-infrared image translation provides a practical way to expand infrared training data using abundant visible images.<n>Existing diffusion-based methods typically use semantic priors only as external conditions.<n>We propose SC-Diff, a semantically latent diffusion framework that uses semantic priors for both conditional guidance and internal self-attention calibration.
  arXiv  Detail & Related papers  (2026-08-09T07:57:40Z)
• BiSLW: Bi-Spectral Latent Watermarking for Generative Diffusion Models [0.0]
  BiSLW is a trainable bi-spectral latent watermarking framework that embeds aligned identity signals across complementary spectral bands.<n>Low-frequency components encode global semantics, while high-frequency components capture fine texture.<n>BiSLW achieves a strong balance between fidelity and robustness, improving PSNR by over 3 dB compared to prior latent diffusion watermarking methods.
  arXiv  Detail & Related papers  (2026-07-02T17:48:54Z)
• Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion [46.51681899437508]
  We introduce Spectral Forcing, a parameter-free, time-conditional 2D-DCT low-pass operator applied to the noisy input before the patch embedder.<n>Its cutoff expands monotonically with the diffusion time and becomes the identity at the data endpoint.<n>On ImageNet-256 with JiT-700M/32, Spectral Forcing consistently improves both FID and Inception Score across different training epochs.
  arXiv  Detail & Related papers  (2026-06-13T10:29:11Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• AdaMerge: Salience-Aware Adaptive Token Merging for Training-Free Acceleration of Vision Transformers [5.739405014622565]
  AdaMerge is a token-merging framework based on two complementary mechanisms.<n>It consistently outperforms ToMe, PiToMe, and DSM across all FLOPs-matched regimes.
  arXiv  Detail & Related papers  (2026-05-26T05:35:39Z)
• CIAR: Interval-based Collaborative Decoding for Image Generation Acceleration [47.52310736101766]
  Auto-regressive (AR) models have recently made notable progress in image generation, achieving performance comparable to diffusion-based approaches.<n>We address this via a cloud-device collaboration framework textbfCIAR, which utilizes on-device self-verification to handle two key properties of visual synthesis.<n>Our solution centers on an on-device token uncertainty quantifier, which adopts continuous probability intervals to accelerate processing and make it feasible for large visual vocabularies.
  arXiv  Detail & Related papers  (2026-03-26T14:04:50Z)
• Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity [54.95089105944234]
  Vision-language models (VLMs) face significant computational inefficiencies caused by excessive generation of visual tokens.<n>We propose PruneSID, a training-free Synergistic Importance-Diversity approach featuring a two-stage pipeline.<n>We demonstrate state-of-the-art performance, achieving 96.3% accuracy on LLaVA-1.5 with only 11.1% token retention, and 92.8% accuracy at extreme compression rates (5.6%) on LLaVA-NeXT.
  arXiv  Detail & Related papers  (2026-03-10T10:31:58Z)
• Making Training-Free Diffusion Segmentors Scale with the Generative Power [118.72472901404814]
  A line of research focuses on adapting a pre-trained diffusion model to semantic segmentation without any further training.<n>We propose two techniques: auto aggregation and per-pixel rescaling, which together enable training-free segmentation to better leverage generative capability.
  arXiv  Detail & Related papers  (2026-03-06T11:35:37Z)
• Towards Efficient Low-rate Image Compression with Frequency-aware Diffusion Prior Refinement [12.819185499129388]
  We propose Accelerate textbfDiffusion-based Image Compression via textbfConsistency Prior textbfRefinement (DiffCR)<n>DiffCR is a novel compression framework for efficient and high-fidelity image reconstruction.
  arXiv  Detail & Related papers  (2026-01-15T13:25:25Z)
• Few-Step Diffusion via Score identity Distillation [67.07985339442703]
  Diffusion distillation has emerged as a promising strategy for accelerating text-to-image (T2I) diffusion models.<n>Existing methods rely on real or teacher-synthesized images to perform well when distilling high-resolution T2I diffusion models.<n>We propose two new guidance strategies: Zero-CFG, which disables CFG in the teacher and removes text conditioning in the fake score network, and Anti-CFG, which applies negative CFG in the fake score network.
  arXiv  Detail & Related papers  (2025-05-19T03:45:16Z)
• CoMatch: Dynamic Covisibility-Aware Transformer for Bilateral Subpixel-Level Semi-Dense Image Matching [31.42896369011162]
  CoMatch is a novel semi-dense image matcher with dynamic covisibility awareness and bilateral subpixel accuracy.<n>A covisibility-guided token condenser is introduced to adaptively aggregate tokens in light of their covisibility scores.<n>A fine correlation module is developed to refine the matching candidates in both source and target views to subpixel level.
  arXiv  Detail & Related papers  (2025-03-31T10:17:01Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Compressing gradients by exploiting temporal correlation in momentum-SGD [17.995905582226463]
  We analyze compression methods that exploit temporal correlation in systems with and without error-feedback. Experiments with the ImageNet dataset demonstrate that our proposed methods offer significant reduction in the rate of communication. We prove the convergence of SGD under an expected error assumption by establishing a bound for the minimum gradient norm.
  arXiv  Detail & Related papers  (2021-08-17T18:04:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.