Ensemble Distribution Distillation for Self-Supervised Human Activity Recognition

• URL: http://arxiv.org/abs/2509.08225v1
• Date: Wed, 10 Sep 2025 01:55:20 GMT
• Title: Ensemble Distribution Distillation for Self-Supervised Human Activity Recognition
• Authors: Matthew Nolan, Lina Yao, Robert Davidson,
• Abstract summary: This paper explores a novel application of Ensemble Distribution Distillation (EDD) within a self-supervised learning framework for Human Activity Recognition (HAR)<n>By leveraging unlabeled data and a partially supervised training strategy, our approach yields an increase in predictive accuracy, robust estimates of uncertainty, and substantial increases in robustness against adversarial perturbation.
• Score: 13.389902854768906
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Human Activity Recognition (HAR) has seen significant advancements with the adoption of deep learning techniques, yet challenges remain in terms of data requirements, reliability and robustness. This paper explores a novel application of Ensemble Distribution Distillation (EDD) within a self-supervised learning framework for HAR aimed at overcoming these challenges. By leveraging unlabeled data and a partially supervised training strategy, our approach yields an increase in predictive accuracy, robust estimates of uncertainty, and substantial increases in robustness against adversarial perturbation; thereby significantly improving reliability in real-world scenarios without increasing computational complexity at inference. We demonstrate this with an evaluation on several publicly available datasets. The contributions of this work include the development of a self-supervised EDD framework, an innovative data augmentation technique designed for HAR, and empirical validation of the proposed method's effectiveness in increasing robustness and reliability.

Related papers

• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data [49.36938105983916]
  Unsupervised domain adaptation methods seek to generalize effectively on unlabeled test data.<n>We propose incorporating multi-scale feature extraction and uncertainty estimation to improve the model's generalization and robustness across domains.
  arXiv  Detail & Related papers  (2025-08-26T03:13:08Z)
• Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation [52.83870601473094]
  Embodied agents exhibit immense potential across a multitude of domains.<n>Existing research predominantly concentrates on the security of general large language models.<n>This paper introduces a novel input moderation framework, meticulously designed to safeguard embodied agents.
  arXiv  Detail & Related papers  (2025-04-22T08:34:35Z)
• Bridging Interpretability and Robustness Using LIME-Guided Model Refinement [0.0]
  Local Interpretable Model-Agnostic Explanations (LIME) systematically enhance model robustness.<n> Empirical evaluations on multiple benchmark datasets demonstrate that LIME-guided refinement not only improves interpretability but also significantly enhances resistance to adversarial perturbations and generalization to out-of-distribution data.
  arXiv  Detail & Related papers  (2024-12-25T17:32:45Z)
• KRAIL: A Knowledge-Driven Framework for Base Human Reliability Analysis Integrating IDHEAS and Large Language Models [2.7378790256389047]
  This paper introduces a novel two-stage framework for knowledge-driven reliability analysis, integrating IDHEAS and LLMs (KRAIL)<n>Inspired by the success of large language models (LLMs) in natural language processing, this paper introduces a novel two-stage framework for knowledge-driven reliability analysis.
  arXiv  Detail & Related papers  (2024-12-20T06:21:34Z)
• Diversified Ensemble of Independent Sub-Networks for Robust Self-Supervised Representation Learning [10.784911682565879]
  Ensembling a neural network is a widely recognized approach to enhance model performance, estimate uncertainty, and improve robustness in deep supervised learning. We present a novel self-supervised training regime that leverages an ensemble of independent sub-networks. Our method efficiently builds a sub-model ensemble with high diversity, leading to well-calibrated estimates of model uncertainty.
  arXiv  Detail & Related papers  (2023-08-28T16:58:44Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• Improving Regression Uncertainty Estimation Under Statistical Change [7.734726150561088]
  We propose and implement a loss function for regression uncertainty estimation based on the Bayesian Validation Metric framework. A series of experiments on in-distribution data show that the proposed method is competitive with existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-09-16T20:32:58Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)
• Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery [85.36948722680822]
  We develop a context-aware mixture of deep models termed the alpha-beta network. We improve accuracy and F score by 10% by identifying high-level contexts. In order to ensure training stability, we have used a clustering-based pre-training in both public and in-house datasets.
  arXiv  Detail & Related papers  (2020-03-03T19:35:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.