Ternary and Binary Quantization for Improved Classification

• URL: http://arxiv.org/abs/2203.16798v1
• Date: Thu, 31 Mar 2022 05:04:52 GMT
• Title: Ternary and Binary Quantization for Improved Classification
• Authors: Weizhi Lu, Mingrui Chen, Kai Guo and Weiyu Li
• Abstract summary: We study the methodology of first reducing data dimension by random projection and then quantizing the projections to ternary or binary codes. We observe that the quantization could provide comparable and often superior accuracy, as the data to be quantized are sparse features generated with common filters.
• Score: 11.510216175832568
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dimension reduction and data quantization are two important methods for reducing data complexity. In the paper, we study the methodology of first reducing data dimension by random projection and then quantizing the projections to ternary or binary codes, which has been widely applied in classification. Usually, the quantization will seriously degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could provide comparable and often superior accuracy, as the data to be quantized are sparse features generated with common filters. Furthermore, this quantization property could be maintained in the random projections of sparse features, if both the features and random projection matrices are sufficiently sparse. By conducting extensive experiments, we validate and analyze this intriguing property.

Related papers

• The Binary and Ternary Quantization Can Improve Feature Discrimination [8.723496120436169]
  In machine learning, quantization is widely used to simplify data representation and facilitate algorithm deployment on hardware. Current research focuses on quantization errors, operating under the premise that higher quantization errors generally result in lower classification performance. We show that certain extremely low bit-width quantization methods, such as $0,1$-binary quantization and $0, pm1$-ternary quantization, can achieve comparable or even superior classification accuracy.
  arXiv  Detail & Related papers  (2025-04-18T16:44:12Z)
• Conformalized High-Density Quantile Regression via Dynamic Prototypes-based Probability Density Estimation [2.526146573337397]
  We introduce a conformalized high-density quantile regression approach with a dynamically adaptive set of prototypes. Our method optimize the set of prototypes by adaptively adding, deleting, and relocating quantization bins. Experiments across diverse datasets and dimensionalities confirm that our method consistently achieves high-quality prediction regions.
  arXiv  Detail & Related papers  (2024-11-02T14:36:12Z)
• Quantization of Large Language Models with an Overdetermined Basis [73.79368761182998]
  We introduce an algorithm for data quantization based on the principles of Kashin representation. Our findings demonstrate that Kashin Quantization achieves competitive or superior quality in model performance.
  arXiv  Detail & Related papers  (2024-04-15T12:38:46Z)
• Implicit Manifold Gaussian Process Regression [49.0787777751317]
  Gaussian process regression is widely used to provide well-calibrated uncertainty estimates. It struggles with high-dimensional data because of the implicit low-dimensional manifold upon which the data actually lies. In this paper we propose a technique capable of inferring implicit structure directly from data (labeled and unlabeled) in a fully differentiable way.
  arXiv  Detail & Related papers  (2023-10-30T09:52:48Z)
• Regularized Vector Quantization for Tokenized Image Synthesis [126.96880843754066]
  Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while quantization suffers from low codebook utilization and reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate perturbed above issues effectively by applying regularization from two perspectives.
  arXiv  Detail & Related papers  (2023-03-11T15:20:54Z)
• Quantum Sparse Coding [5.130440339897477]
  We develop a quantum-inspired algorithm for sparse coding. The emergence of quantum computers and Ising machines can potentially lead to more accurate estimations. We conduct numerical experiments with simulated data on Lightr's quantum-inspired digital platform.
  arXiv  Detail & Related papers  (2022-09-08T13:00:30Z)
• Improved Quantum Algorithms for Fidelity Estimation [77.34726150561087]
  We develop new and efficient quantum algorithms for fidelity estimation with provable performance guarantees. Our algorithms use advanced quantum linear algebra techniques, such as the quantum singular value transformation. We prove that fidelity estimation to any non-trivial constant additive accuracy is hard in general.
  arXiv  Detail & Related papers  (2022-03-30T02:02:16Z)
• High Dimensional Statistical Estimation under One-bit Quantization [27.718986773043643]
  One-bit (binary) data are preferable in many applications because of the efficiency in signal storage, processing, transmission, and enhancement of privacy. In this paper, we study three fundamental statistical estimation problems. Under both sub-Gaussian and heavy-tailed regimes, new estimators that handle high-dimensional scaling are proposed.
  arXiv  Detail & Related papers  (2022-02-26T15:13:04Z)
• Dual-Frequency Quantum Phase Estimation Mitigates the Spectral Leakage of Quantum Algorithms [76.15799379604898]
  Quantum phase estimation suffers from spectral leakage when the reciprocal of the record length is not an integer multiple of the unknown phase. We propose a dual-frequency estimator, which approaches the Cramer-Rao bound, when multiple samples are available.
  arXiv  Detail & Related papers  (2022-01-23T17:20:34Z)
• Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss [61.26793005355441]
  Cluster-Promoting Quantization (CPQ) finds the optimal quantization grids for neural networks. DropBits is a new bit-drop technique that revises the standard dropout regularization to randomly drop bits instead of neurons. We experimentally validate our method on various benchmark datasets and network architectures.
  arXiv  Detail & Related papers  (2021-09-05T15:15:07Z)
• Regularized Classification-Aware Quantization [39.04839665081476]
  We present a class of algorithms that learn distributed quantization schemes for binary classification tasks. Our method is called Regularized Classification-Aware Quantization.
  arXiv  Detail & Related papers  (2021-07-12T21:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.