Fast-HaMeR: Boosting Hand Mesh Reconstruction using Knowledge Distillation

• URL: http://arxiv.org/abs/2603.16444v1
• Date: Tue, 17 Mar 2026 12:28:02 GMT
• Title: Fast-HaMeR: Boosting Hand Mesh Reconstruction using Knowledge Distillation
• Abstract summary: 3D hand reconstruction is essential for real-time applications in VR/AR, human-computer interaction, robotics, and healthcare.<n>Most state-of-the-art methods rely on heavy models, limiting their use on resource-constrained devices like headsets, smartphones, and embedded systems.<n>In this paper, we investigate how the use of lightweight neural networks, combined with Knowledge Distillation, can accelerate complex 3D hand reconstruction models.
• Score: 14.044063434854166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fast and accurate 3D hand reconstruction is essential for real-time applications in VR/AR, human-computer interaction, robotics, and healthcare. Most state-of-the-art methods rely on heavy models, limiting their use on resource-constrained devices like headsets, smartphones, and embedded systems. In this paper, we investigate how the use of lightweight neural networks, combined with Knowledge Distillation, can accelerate complex 3D hand reconstruction models by making them faster and lighter, while maintaining comparable reconstruction accuracy. While our approach is suited for various hand reconstruction frameworks, we focus primarily on boosting the HaMeR model, currently the leading method in terms of reconstruction accuracy. We replace its original ViT-H backbone with lighter alternatives, including MobileNet, MobileViT, ConvNeXt, and ResNet, and evaluate three knowledge distillation strategies: output-level, feature-level, and a hybrid of both. Our experiments show that using lightweight backbones that are only 35% the size of the original achieves 1.5x faster inference speed while preserving similar performance quality with only a minimal accuracy difference of 0.4mm. More specifically, we show how output-level distillation notably improves student performance, while feature-level distillation proves more effective for higher-capacity students. Overall, the findings pave the way for efficient real-world applications on low-power devices. The code and models are publicly available under https://github.com/hunainahmedj/Fast-HaMeR.

Related papers

• Lite Any Stereo V2: Faster and Stronger Efficient Zero-Shot Stereo Matching [96.63979376005277]
  Lite Any Stereo V2 is an ultra-fast model series designed for efficient zero-shot stereo matching.<n>LAS2 is developed from both architecture and training perspectives.<n>LAS2-H achieves stronger overall zero-shot performance than the iterative method Fast-FoundationStereo.
  arXiv  Detail & Related papers  (2026-06-23T11:45:09Z)
• Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matching [16.927491376135134]
  We present Fast-FoundationStereo, a family of architectures that achieve, for the first time, strong zero-shot generalization at real-time frame rate.<n>We employ a divide-and-conquer acceleration strategy with three components: knowledge distillation, blockwise neural architecture search and structured pruning.<n>The resulting model can run over 10x faster than FoundationStereo while closely matching its zero-shot accuracy.
  arXiv  Detail & Related papers  (2025-12-11T21:36:29Z)
• PocketSR: The Super-Resolution Expert in Your Pocket Mobiles [69.26751136689533]
  Real-world image super-resolution (RealSR) aims to enhance the visual quality of in-the-wild images, such as those captured by mobile phones.<n>Existing methods leveraging large generative models demonstrate impressive results, but the high computational cost and latency make them impractical for edge deployment.<n>We introduce PocketSR, an ultra-lightweight, single-step model that brings generative modeling capabilities to RealSR while maintaining high fidelity.
  arXiv  Detail & Related papers  (2025-10-03T13:56:18Z)
• SD3.5-Flash: Distribution-Guided Distillation of Generative Flows [87.45964232927945]
  We present SD3.5-Flash, an efficient few-step distillation framework that brings high-quality image generation to consumer devices.<n>We introduce two key innovations: "timestep sharing" to reduce gradient noise and "split-timestep fine-tuning" to improve prompt alignment.<n>This democratizes access across the full spectrum of devices, from mobile phones to desktop computers.
  arXiv  Detail & Related papers  (2025-09-25T16:07:38Z)
• GhostNetV3-Small: A Tailored Architecture and Comparative Study of Distillation Strategies for Tiny Images [0.0]
  This paper explores strategies for compressing and adapting models to enable efficient inference in resource-constrained environments.<n>We focus on GhostNetV3, a state-of-the-art architecture for mobile applications, and propose GhostNetV3-Small, a modified variant designed to perform better on low-resolution inputs.
  arXiv  Detail & Related papers  (2025-09-15T19:19:09Z)
• FastFit: Accelerating Multi-Reference Virtual Try-On via Cacheable Diffusion Models [59.8871829077739]
  FastFit is a high-speed multi-reference virtual try-on framework based on a novel cacheable diffusion architecture.<n>Our model fully decouples reference feature encoding from the denoising process with negligible parameter overhead.<n>This allows reference features to be computed only once and losslessly reused across all steps.
  arXiv  Detail & Related papers  (2025-08-28T09:25:52Z)
• Topology-Guided Knowledge Distillation for Efficient Point Cloud Processing [3.3903891679981593]
  This work introduces a novel distillation framework to transfer knowledge from a high-capacity teacher to a lightweight student model.<n>Our approach captures the underlying geometric structures of point clouds while selectively guiding the student model's learning process.<n>Our method achieves state-of-the-art performance among knowledge distillation techniques trained solely on LiDAR data.
  arXiv  Detail & Related papers  (2025-05-12T22:15:54Z)
• HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference [54.40808356999408]
  We present HOBBIT, a mixed precision expert offloading system to enable flexible and efficient MoE inference. Our key insight is that dynamically replacing less critical cache-miss experts with low precision versions can substantially reduce expert-loading latency. HOBBIT achieves up to a 9.93x speedup in decoding compared to state-of-the-art MoE offloading systems.
  arXiv  Detail & Related papers  (2024-11-03T04:25:46Z)
• Distill-then-prune: An Efficient Compression Framework for Real-time Stereo Matching Network on Edge Devices [5.696239274365031]
  We propose a novel strategy by incorporating knowledge distillation and model pruning to overcome the inherent trade-off between speed and accuracy. We obtained a model that maintains real-time performance while delivering high accuracy on edge devices.
  arXiv  Detail & Related papers  (2024-05-20T06:03:55Z)
• Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation [56.053397775016755]
  We propose a sequential approach to knowledge distillation that progressively transfers the knowledge of a set of teacher detectors to a given lightweight student. To the best of our knowledge, we are the first to successfully distill knowledge from Transformer-based teacher detectors to convolution-based students.
  arXiv  Detail & Related papers  (2023-08-17T17:17:08Z)
• FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization [14.707312504365376]
  We introduce FastViT, a hybrid vision transformer architecture that obtains the state-of-the-art latency-accuracy trade-off. We show that our model is 3.5x faster than CMT, 4.9x faster than EfficientNet, and 1.9x faster than ConvNeXt on a mobile device for the same accuracy on the ImageNet dataset.
  arXiv  Detail & Related papers  (2023-03-24T17:58:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.