Benchmarking Federated Learning and Knowledge Distillation for Point Cloud Classification

• URL: http://arxiv.org/abs/2607.01272v1
• Date: Tue, 30 Jun 2026 20:12:27 GMT
• Title: Benchmarking Federated Learning and Knowledge Distillation for Point Cloud Classification
• Abstract summary: We present a benchmark jointly evaluating federated learning (FL) and knowledge distillation (KD) for 3D point cloud classification.<n>It spans 13 FL algorithms and 10 KD objectives (a 130-pair cross-product) across 504 training runs, evaluated on ModelNet40 and a clinical craniosynostosis dataset.
• Score: 5.584060970507507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying 3D point cloud analysis in privacy-sensitive, resource-constrained settings faces two barriers: data cannot be centralized, and models must run on limited edge hardware. We present a multi-seed benchmark jointly evaluating federated learning (FL) and knowledge distillation (KD) for 3D point cloud classification. It spans 13 FL algorithms and 10 KD objectives (a 130-pair cross-product) across 504 training runs, evaluated on ModelNet40 and a clinical craniosynostosis dataset. We report three findings. First, under extreme non-IID label skew, standalone FL degrades sharply: on ModelNet40, the strongest method reaches 76.32% against a 92.26% centralized reference; on clinical data, the best reaches 75.83% against 100%. Second, distillation successfully compresses the teacher into a student 74.51% smaller and roughly twice as fast at inference, often matching or surpassing the teacher. Third, the combined pipeline exposes an evaluation pitfall: when distillation keeps a hard-label cross-entropy term on a labeled proxy split, a collapsed federated teacher (8.50%) paired with Logit-MSE still yields a 92.94% student. This 84.4-point gap reflects the proxy labels rather than the federated model, reusing the very labels whose privacy motivated federation. Objectives without hard labels instead track teacher quality ($r \approx 0.99$) and collapse when the teacher does. We therefore recommend evaluating FL-KD pipelines with label-free distillation so reported accuracy reflects the federated teacher, not the proxy.

Related papers

• Cross-Architecture Knowledge Distillation from a Vision Foundation Model to a Lightweight Visual State Space Model for Tea Leaf Disease Classification [10.122947222129108]
  Self-supervised vision foundation models such as DINOv2 provide strong features but are too large for field deployment.<n>We study cross-architecture knowledge distillation from a fine-tuned DINOv2 teacher to a compact bidirectional Visual State Space Model.<n>We identify and fix two training-stability problems that prevent the from-scratch SSM student from learning on limited data.
  arXiv  Detail & Related papers  (2026-08-27T08:02:05Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation [81.10000755917712]
  On-policy distillation transfers reasoning capabilities by training a student model on its own generated trajectories using token-level feedback from a teacher.<n>As student-generated prefixes lengthen, the teacher's next-token distribution becomes less confident and less discriminative.<n>To mitigate SFD, we introduce textbfLookahead Group Reward (ours).
  arXiv  Detail & Related papers  (2026-05-29T04:39:20Z)
• Asking Back: Interaction-Layer Antidistillation Watermarks [7.826668598190874]
  Existing defenses operate on the teacher's output tokens.<n>Recent work shows a oblivious attacker can strip these signals without losing the underlying knowledge.<n>We propose interaction-layer antidistillation watermarks.
  arXiv  Detail & Related papers  (2026-05-15T08:28:35Z)
• DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation [49.98710755440242]
  Distillation enables compact Vision-Language Models (VLMs) to obtain strong reasoning capabilities.<n>We reveal a critical inefficiency in this approach: up to 69% of the prompts in standard chart / document reasoning datasets are effectively zero-delta.<n>We propose a staged synthesis pipeline that repurposes existing datasets as seeds, actively targeting student failure modes to produce better prompts.
  arXiv  Detail & Related papers  (2026-05-15T02:04:12Z)
• Cognitive-Uncertainty Guided Knowledge Distillation for Accurate Classification of Student Misconceptions [11.509375725817195]
  We propose a two-stage knowledge distillation framework that mines high-value samples from existing data.<n>With augmented training on only 10.30% of filtered samples, we achieve MAP@3 of 0.9585 (+17.8%) on the MAP-Charting dataset.<n>Using only a 4B parameter model, we attain 84.38% accuracy on cross-topic tests of middle school algebra misconception benchmarks.
  arXiv  Detail & Related papers  (2026-05-14T12:17:38Z)
• When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models [18.966595547209824]
  We propose a Hybrid Kimi Delta Attention (Hybrid-KDA) architecture paired with GenDistill.<n>We find that log-likelihood-based evaluation consistently underestimates the gap between teacher and student.
  arXiv  Detail & Related papers  (2026-03-27T16:16:23Z)
• LGD: Label-guided Self-distillation for Object Detection [59.9972914042281]
  We propose the first self-distillation framework for general object detection, termed LGD (Label-Guided self-Distillation) Our framework involves sparse label-appearance encoding, inter-object relation adaptation and intra-object knowledge mapping to obtain the instructive knowledge. Compared with a classical teacher-based method FGFI, LGD not only performs better without requiring pretrained teacher but also with 51% lower training cost beyond inherent student learning.
  arXiv  Detail & Related papers  (2021-09-23T16:55:01Z)
• G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation [49.421099172544196]
  We propose a novel semantic-guided feature imitation technique, which automatically performs soft matching between feature pairs across all pyramid levels. We also introduce contrastive distillation to effectively capture the information encoded in the relationship between different feature regions. Our method consistently outperforms the existing detection KD techniques, and works when (1) components in the framework are used separately and in conjunction.
  arXiv  Detail & Related papers  (2021-08-17T07:44:27Z)
• Humble Teachers Teach Better Students for Semi-Supervised Object Detection [7.764145630268344]
  Our model achieves COCO-style AP of 53.04% on VOC07 val set, 8.4% better than STAC, when using VOC12 as unlabeled data. It also reaches 53.8% AP on MS-COCO test-dev with 3.1% gain over the fully supervised ResNet-152 Cascaded R-CNN.
  arXiv  Detail & Related papers  (2021-06-19T09:05:10Z)
• 3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection [76.42897462051067]
  3DIoUMatch is a novel semi-supervised method for 3D object detection applicable to both indoor and outdoor scenes. We leverage a teacher-student mutual learning framework to propagate information from the labeled to the unlabeled train set in the form of pseudo-labels. Our method consistently improves state-of-the-art methods on both ScanNet and SUN-RGBD benchmarks by significant margins under all label ratios.
  arXiv  Detail & Related papers  (2020-12-08T11:06:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.