Ef-QuantFace: Streamlined Face Recognition with Small Data and Low-Bit Precision

• URL: http://arxiv.org/abs/2402.18163v1
• Date: Wed, 28 Feb 2024 08:53:01 GMT
• Title: Ef-QuantFace: Streamlined Face Recognition with Small Data and Low-Bit Precision
• Authors: William Gazali, Jocelyn Michelle Kho, Joshua Santoso, Williem
• Abstract summary: This paper introduces an efficiency-driven approach, fine-tuning the model with just up to 14,000 images, 440 times smaller than MS1M. We demonstrate that effective quantization is achievable with a smaller dataset, presenting a new paradigm.
• Score: 1.9389881806157312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, model quantization for face recognition has gained prominence. Traditionally, compressing models involved vast datasets like the 5.8 million-image MS1M dataset as well as extensive training times, raising the question of whether such data enormity is essential. This paper addresses this by introducing an efficiency-driven approach, fine-tuning the model with just up to 14,000 images, 440 times smaller than MS1M. We demonstrate that effective quantization is achievable with a smaller dataset, presenting a new paradigm. Moreover, we incorporate an evaluation-based metric loss and achieve an outstanding 96.15% accuracy on the IJB-C dataset, establishing a new state-of-the-art compressed model training for face recognition. The subsequent analysis delves into potential applications, emphasizing the transformative power of this approach. This paper advances model quantization by highlighting the efficiency and optimal results with small data and training time.

Related papers

• HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence Modeling [52.58723853697152]
  We propose a Hybrid Architecture Distillation (HAD) approach for DNA sequence modeling.<n>We employ the NTv2-500M as the teacher model and devise a grouping masking strategy.<n>Compared to models with similar parameters, our model achieved excellent performance.
  arXiv  Detail & Related papers  (2025-05-27T07:57:35Z)
• On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models [27.402312440922557]
  Key components of the best performing latent diffusion models (LDMs) training recipes are oftentimes not available to the research community. We perform an in-depth study of LDM training recipes focusing on the performance of models and their training efficiency. We propose a novel conditioning mechanism that disentangles semantic and control metadata conditionings.
  arXiv  Detail & Related papers  (2024-11-05T15:22:26Z)
• Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matching [74.75248610868685]
  Teddy is a Taylor-approximated dataset distillation framework designed to handle large-scale dataset. Teddy attains state-of-the-art efficiency and performance on the Tiny-ImageNet and original-sized ImageNet-1K dataset.
  arXiv  Detail & Related papers  (2024-10-10T03:28:46Z)
• A Simple and Efficient Baseline for Data Attribution on Images [107.12337511216228]
  Current state-of-the-art approaches require a large ensemble of as many as 300,000 models to accurately attribute model predictions. In this work, we focus on a minimalist baseline, utilizing the feature space of a backbone pretrained via self-supervised learning to perform data attribution. Our method is model-agnostic and scales easily to large datasets.
  arXiv  Detail & Related papers  (2023-11-03T17:29:46Z)
• Efficiently Robustify Pre-trained Models [18.392732966487582]
  robustness of large scale models towards real-world settings is still a less-explored topic. We first benchmark the performance of these models under different perturbations and datasets. We then discuss on how complete model fine-tuning based existing robustification schemes might not be a scalable option given very large scale networks.
  arXiv  Detail & Related papers  (2023-09-14T08:07:49Z)
• Sample Less, Learn More: Efficient Action Recognition via Frame Feature Restoration [59.6021678234829]
  We propose a novel method to restore the intermediate features for two sparsely sampled and adjacent video frames. With the integration of our method, the efficiency of three commonly used baselines has been improved by over 50%, with a mere 0.5% reduction in recognition accuracy.
  arXiv  Detail & Related papers  (2023-07-27T13:52:42Z)
• Delving Deeper into Data Scaling in Masked Image Modeling [145.36501330782357]
  We conduct an empirical study on the scaling capability of masked image modeling (MIM) methods for visual recognition. Specifically, we utilize the web-collected Coyo-700M dataset. Our goal is to investigate how the performance changes on downstream tasks when scaling with different sizes of data and models.
  arXiv  Detail & Related papers  (2023-05-24T15:33:46Z)
• The effectiveness of MAE pre-pretraining for billion-scale pretraining [65.98338857597935]
  We introduce an additional pre-pretraining stage that is simple and uses the self-supervised MAE technique to initialize the model. We measure the effectiveness of pre-pretraining on 10 different visual recognition tasks spanning image classification, video recognition, object detection, low-shot classification and zero-shot recognition.
  arXiv  Detail & Related papers  (2023-03-23T17:56:12Z)
• On Data Scaling in Masked Image Modeling [36.00347416479826]
  Masked image modeling (MIM) is suspected to be unable to benefit from larger data. Data scales ranging from 10% of ImageNet-1K to full ImageNet-22K, model sizes ranging from 49 million to 1 billion, and training lengths ranging from 125K iterations to 500K iterations. validation loss in pre-training is a good indicator to measure how well the model performs for fine-tuning on multiple tasks.
  arXiv  Detail & Related papers  (2022-06-09T17:58:24Z)
• STAR: Sparse Transformer-based Action Recognition [61.490243467748314]
  This work proposes a novel skeleton-based human action recognition model with sparse attention on the spatial dimension and segmented linear attention on the temporal dimension of data. Experiments show that our model can achieve comparable performance while utilizing much less trainable parameters and achieve high speed in training and inference.
  arXiv  Detail & Related papers  (2021-07-15T02:53:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.