dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats

• URL: http://arxiv.org/abs/2606.04115v1
• Date: Tue, 02 Jun 2026 18:23:20 GMT
• Title: dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats
• Abstract summary: dMX is a mixed-precision quantization framework for learnable floating-point bit-width assignment.<n>We study its application for the microscaling floating-point (MXFP) family of data types defined by the Open Compute Project standard.
• Score: 1.3357020021155022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantizing large language models (LLMs) to low-precision floating-point representations is central to efficient deployment, yet applying a single bit-width uniformly across all layers is sub-optimal in terms of both performance and accuracy. This work introduces dMX, a differentiable mixed-precision quantization framework for learnable floating-point bit-width assignment. We study its application for the microscaling floating-point (MXFP) family of data types defined by the Open Compute Project (OCP) standard. The per-layer bit-width assignment is formulated as a continuous optimization problem in which each layer's floating-point format format is parameterized by a scalar parameter, folding the multi-variate design space into a single learnable offset. During training this offset takes continuous values, avoiding sudden oscillations between discrete quantization formats. A temperature-based annealing schedule progressively discretizes the learned offsets, ensuring that the final configuration maps to hardware-compatible MXFP formats without abrupt transitions between training and inference behavior. A target-aware regularization term steers the average bit-width toward a user-specified budget, serving as a coarse-grained proxy for inference cost and balancing model quality against deployment efficiency. We performed experiments on different families of LLM, such as Llama, Qwen3, and SmolLM2, evaluating perplexity on WikiText-2 and accuracy on four zero-shot reasoning benchmarks. Across these settings, dMX consistently yields Pareto-dominating models and improves over Kullback-Leibler (KL) divergence-based layer-selection heuristics, efficiently navigating trade-offs between model quality and average bit-width.

Related papers

• Variable-Width Transformers [57.07167443557886]
  We investigate nonuniform capacity allocation across network depth by proposing a $times$-shaped > former architecture.<n>Our design maintains wider early and late layers while narrowing the middle layers, utilizing a parameter-free residual resizing mechanism.<n>By reducing the average layer width, this architecture also requires fewer overall FLOPs and smaller KV cache memory and I/O cost.
  arXiv  Detail & Related papers  (2026-06-16T17:59:03Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• Multi-Fidelity Flow Matching: Cascaded Refinement of PDE Solutions [6.144605787421899]
  Multi-Fidelity Flow Matching is a cascade refinement framework for PDE solutions.<n>We exploit this in Multi-Fidelity Flow Matching (MFFM), a cascade refinement framework for PDE solutions.<n>MFFM is a learned analog of multigrid refinement that reaches the finest grid in $L$ deterministic network evaluations per query.
  arXiv  Detail & Related papers  (2026-05-15T16:02:18Z)
• Depth Completion as Parameter-Efficient Test-Time Adaptation [66.72360181325877]
  CAPA is a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion.<n>For videos, CAPA introduces sequence-level parameter sharing, jointly adapting all frames to exploit temporal correlations, improve robustness, and enforce multi-frame consistency.
  arXiv  Detail & Related papers  (2026-02-16T13:53:23Z)
• DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment [4.881985877863507]
  DP-LLM is a novel mechanism that dynamically assigns precision to each layer based on input values.<n>We show that DP-LLM achieves a superior performance-latency trade-off, outperforming prior approaches.
  arXiv  Detail & Related papers  (2025-08-08T05:57:04Z)
• Characterization and Mitigation of Training Instabilities in Microscaling Formats [6.025438902954768]
  Training large language models is an expensive, compute-bound process.<n>Next-generation hardware accelerators increasingly support lower-precision arithmetic formats.<n>We investigate the challenges and viability of block-scaled precision formats during model training.
  arXiv  Detail & Related papers  (2025-06-25T18:25:08Z)
• Recipes for Pre-training LLMs with MXFP8 [0.08732752045535426]
  We present results using models with up to 8B parameters, trained on high-quality datasets of up to 15T tokens.<n>We show how MXFP8-E4M3 datatype and a specific number conversion algorithm result in training sessions that match those carried out in BF16.
  arXiv  Detail & Related papers  (2025-05-30T21:08:15Z)
• Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Model [0.0]
  Mix-QSAM is a mixed-precision Post-Training Quantization (PTQ) framework for the Segment Anything Model (SAM)<n>We introduce a layer-wise importance score, derived using Kullback-Leibler (KL) divergence, to quantify each layer's contribution to the model's output.<n>We also introduce cross-layer synergy, a novel metric based on causal mutual information, to capture dependencies between adjacent layers.
  arXiv  Detail & Related papers  (2025-05-08T00:08:31Z)
• Pushing the Limits of Large Language Model Quantization via the Linearity Theorem [71.3332971315821]
  We present a "line theoremarity" establishing a direct relationship between the layer-wise $ell$ reconstruction error and the model perplexity increase due to quantization. This insight enables two novel applications: (1) a simple data-free LLM quantization method using Hadamard rotations and MSE-optimal grids, dubbed HIGGS, and (2) an optimal solution to the problem of finding non-uniform per-layer quantization levels.
  arXiv  Detail & Related papers  (2024-11-26T15:35:44Z)
• Progressive Mixed-Precision Decoding for Efficient LLM Inference [49.05448842542558]
  We introduce Progressive Mixed-Precision Decoding (PMPD) to address the memory-boundedness of decoding.<n>PMPD achieves 1.4$-$12.2$times$ speedup in matrix-vector multiplications over fp16 models.<n>Our approach delivers a throughput gain of 3.8$-$8.0$times$ over fp16 models and up to 1.54$times$ over uniform quantization approaches.
  arXiv  Detail & Related papers  (2024-10-17T11:46:33Z)
• Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences [49.14535254003683]
  We introduce PaLoRA, a novel parameter-efficient method that addresses multi-task trade-offs in machine learning.<n>Our experiments show that PaLoRA outperforms state-of-the-art MTL and PFL baselines across various datasets.
  arXiv  Detail & Related papers  (2024-07-10T21:25:51Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [63.118592279833656]
  Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs)<n>We propose SliM-LLM, a salience-driven mixed-precision quantization framework that allocates bit-widths at the group-wise.<n> Experiments show that SliM-LLM achieves superior performance across various LLMs at low bit-widths.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programming [7.0146264551420066]
  Quantization is a widely used technique to compress neural networks.<n>MPQ addresses this by assigning varied bit-widths to layers, optimizing the accuracy-efficiency trade-off.<n>We introduce CLADO, a practical sensitivity-based MPQ algorithm that captures crosslayer dependency of quantization error.
  arXiv  Detail & Related papers  (2023-07-11T15:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.