W4A4 Quantization for Inference on Wan2.2-I2V-A14B

• URL: http://arxiv.org/abs/2606.29337v1
• Date: Sun, 28 Jun 2026 11:12:52 GMT
• Title: W4A4 Quantization for Inference on Wan2.2-I2V-A14B
• Abstract summary: Sub-challenge targets 4-bit weight and 4-bit activation inference on Wan-AI/Wan2.2-I2V-A14B under HiF4 or MXFP4 numerical formats.<n>We adapt two complementary ideas from LLM quantization, MixQ-style mixed precision for sparse activation outliers and SmoothQuant-style per-channel smoothing.<n>On official VBench I2V metrics, our pipeline stays within 2-3.5 percent of FP16 on most quality axes and improves motion smoothness.
• Score: 23.122181646076882
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We summarize our submission to Sub-Challenge 1: W4A4 Quantization for Inference (HiF4 / MXFP4) of the ICME 2026 Low-Bit-width Large-Model Quantization Challenge. The sub-challenge targets 4-bit weight and 4-bit activation inference on Wan-AI/Wan2.2-I2V-A14B under HiF4 or MXFP4 numerical formats. We adapt two complementary ideas from LLM quantization, MixQ-style mixed precision for sparse activation outliers and SmoothQuant-style per-channel smoothing, together with block-wise HiF4 packing for Wan2.2 feed-forward linear layers. Calibration on representative OpenS2V-5M batches identifies heavy-tailed activation channels; smoothing rebalances dynamic range before W4A4 rounding; and a dual-branch GEMM preserves outlier columns in higher precision while the bulk of channels use strict W4A4. On official VBench I2V metrics, our pipeline stays within 2-3.5 percent of FP16 on most quality axes and improves motion smoothness, outperforming a native HiFloat4 baseline that degrades roughly 5 percent relative to FP16 across all reported scores.

Related papers

• HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models [9.149747042283197]
  We present the first end-to-end FP4 RL post-training, in which both the rollout and training policies operate at 4-bit precision.<n>A lightweight correction, Rollout-ResQ, recovers most of the precision lost to outlier-driven underflow without inflating the rollout's compute footprint.
  arXiv  Detail & Related papers  (2026-07-29T06:28:26Z)
• QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides [26.939321070753607]
  Rollout generation is a major bottleneck in Reinforcement Learning for Mixture-of-Experts (MoE) Large Language Models.<n>We propose QUantization-error Alignment across Dual Sides (QUADS) to stabilize NVFP4 RL for MoE.<n>QUADS achieves BF16-level accuracy, improves average pass@1 by 21.49 points over naive NVFP4 RL, and delivers 16% higher rollout throughput than FP8.
  arXiv  Detail & Related papers  (2026-07-17T10:21:27Z)
• FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models [50.66675248468456]
  FourTune is an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm.<n>FourTune reduces memory overhead by 2.25$times$ and increases end-to-end training throughput by 2.27$times$ compared to BF16 LoRA.
  arXiv  Detail & Related papers  (2026-07-07T00:34:30Z)
• SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference [4.219937026460372]
  SharQ bridges activation sparsity and FP4 quantization through an online sparse--dense decomposition.<n>A sparse FP4 GEMM processes the backbone while a dense FP4 GEMM compensates for both mask-induced activation loss and sparse-path quantization error.
  arXiv  Detail & Related papers  (2026-06-25T04:19:04Z)
• MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models [57.87791117741788]
  We propose a modality-aware PTQ framework to preserve cross-modal morphology and mitigate outlier loss.<n>Distribution-Aware Bias Compensation (DABC) selectively absorbs long-tailed outliers into channel-wise biases.<n>We also propose Morphology-Directed Quantization Function Optimization (MDQFO) to co-optimize the quantization grid with the bias mask.
  arXiv  Detail & Related papers  (2026-06-03T02:05:10Z)
• Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference [3.308743964406687]
  Two-bit weight quantization is attractive for memory-efficient LLM inference.<n>The standard W2 level set -2,-1,0,+1 often collapses under aggressive W2A4/KV4 settings.<n>We propose Qift, a fixed no-zero W2 level set for rotated W2A4/KV4 inference.
  arXiv  Detail & Related papers  (2026-06-01T19:40:32Z)
• Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V [23.570323694430538]
  W4A4 quantization of large video diffusion Transformers offers substantial memory savings but is hindered by two main challenges.<n>These difficulties are compounded by Wan2.2-I2V's two-expert Mixture-of-Experts DiT design.<n>We propose a post-training quantization framework combining SVDQuant-based low-rank outlier compensation, GPTQ-based reconstruction-aware residual weight quantization, and timestep-bin-wise per-layer activation clipping-ratio search conducted independently for each expert.
  arXiv  Detail & Related papers  (2026-05-26T13:24:01Z)
• Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2 [45.26076445356586]
  This report describes Tail-Aware HiFloat4, our submission to the low-bit text-to-video generation quantization challenge.<n>We quantize the main linear layers in both Wan2.2 transformer modules with W4A4 HiFloat4 fake quantization.<n>This design reduces the influence of rare outlier calibrations while keeping the runtime HiFloat4 arithmetic and sampling pipeline unchanged.
  arXiv  Detail & Related papers  (2026-05-26T07:04:22Z)
• DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers [3.0583214514538084]
  Diffusion Transformers (DiTs) achieve state-of-the-art image generation quality but incur substantial memory and computational costs at inference.<n>Existing approaches, including smoothing-based methods, mixed-precision schemes, rotation techniques, and low-rank residual methods, partially mitigate this issue but still leave a noticeable gap to FP16/BF16 performance.<n>We introduce DiRotQ, a W4A4 PTQ framework that mitigates this degradation through rotation-aware activation quantization.
  arXiv  Detail & Related papers  (2026-05-16T00:52:00Z)
• DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization [47.19478866645546]
  We propose DuQuant++, which adapts the outlier-aware fine-grained rotation of DuQuant to the MXFP4 format.<n>Experiments on the LLaMA-3 family under MXFP4 W4A4 quantization show that DuQuant++ consistently achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T04:27:28Z)
• Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization [77.67818998672516]
  We present the first comprehensive study of MXFP4 and NVFP4 for post-training quantization.<n>We introduce Micro-Rotated-GPTQ (MR-GPTQ), a variant of the classic GPTQ quantization algorithm.<n>We show that MR-GPTQ matches or outperforms state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2025-09-27T09:22:21Z)
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models [61.474101404805545]
  Diffusion models can generate high-quality images, but as they scale, rising memory demands and higher latency pose deployment challenges.<n>We propose SVDQuant, a new 4-bit quantization paradigm to overcome this limitation.<n>We reduce the memory usage for the 12B FLUX.1 models by 3.5$times$, achieving 3.0$times$ speedup over the 4-bit weight-only quantization (W4A16) baseline.
  arXiv  Detail & Related papers  (2024-11-07T18:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.