On the Evaluation Protocol of Gesture Recognition for UAV-based Rescue Operation based on Deep Learning: A Subject-Independence Perspective

• URL: http://arxiv.org/abs/2602.17854v1
• Date: Thu, 19 Feb 2026 21:37:42 GMT
• Title: On the Evaluation Protocol of Gesture Recognition for UAV-based Rescue Operation based on Deep Learning: A Subject-Independence Perspective
• Abstract summary: We show that the reported near-perfect accuracy metrics result from a frame-level random train-test split.<n>We demonstrate that the evaluation does not measure generalization to unseen individuals.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a methodological analysis of the gesture-recognition approach proposed by Liu and Szirányi, with a particular focus on the validity of their evaluation protocol. We show that the reported near-perfect accuracy metrics result from a frame-level random train-test split that inevitably mixes samples from the same subjects across both sets, causing severe data leakage. By examining the published confusion matrix, learning curves, and dataset construction, we demonstrate that the evaluation does not measure generalization to unseen individuals. Our findings underscore the importance of subject-independent data partitioning in vision-based gesture-recognition research, especially for applications - such as UAV-human interaction - that require reliable recognition of gestures performed by previously unseen people.

Related papers

• Can Large Language Models Detect Methodological Flaws? Evidence from Gesture Recognition for UAV-Based Rescue Operation Based on Deep Learning [0.0]
  We analyze a gesture-recognition paper reporting near-perfect accuracy on a small, human-centered dataset.<n>We first show that the evaluation protocol is consistent with subject-level data leakage due to non-independent training and test splits.<n>We then assess whether this flaw can be detected independently by six state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2026-03-23T11:39:16Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Learning from Multiple Expert Annotators for Enhancing Anomaly Detection in Medical Image Analysis [0.31317409221921133]
  In medical imaging analysis, multiple expert annotators often produce subjective estimates about "ground truth labels" We propose a simple yet effective approach to combine annotations from multiple radiology experts for training a deep learning-based detector.
  arXiv  Detail & Related papers  (2022-03-20T17:57:26Z)
• Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes [65.91730154730905]
  In applications of offline reinforcement learning to observational data, such as in healthcare or education, a general concern is that observed actions might be affected by unobserved factors. Here we tackle this by considering off-policy evaluation in a partially observed Markov decision process (POMDP) We extend the framework of proximal causal inference to our POMDP setting, providing a variety of settings where identification is made possible.
  arXiv  Detail & Related papers  (2021-10-28T17:46:14Z)
• Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization [77.8735802150511]
  We propose a cross-sample adversarial debiasing (CSAD) method to remove the bias information misused by the target task. The correlation measurement plays a critical role in adversarial debiasing and is conducted by a cross-sample neural mutual information estimator. We conduct thorough experiments on publicly available datasets to validate the advantages of the proposed method over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-08-11T21:17:02Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Evaluation of Unsupervised Entity and Event Salience Estimation [17.74208462902158]
  Salience Estimation aims to predict term importance in documents. Previous studies typically generate pseudo-ground truth for evaluation. In this work, we propose a light yet practical entity and event salience estimation evaluation protocol.
  arXiv  Detail & Related papers  (2021-04-14T15:23:08Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)
• Evaluation Discrepancy Discovery: A Sentence Compression Case-study [0.6430130814523793]
  We show that neither metric nor human evaluation is sufficient to draw conclusions about system performance. We demonstrate how a system can game a well-established dataset to achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2021-01-22T12:28:41Z)
• On the uncertainty of self-supervised monocular depth estimation [52.13311094743952]
  Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all. We explore for the first time how to estimate the uncertainty for this task and how this affects depth accuracy. We propose a novel peculiar technique specifically designed for self-supervised approaches.
  arXiv  Detail & Related papers  (2020-05-13T09:00:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.