How Much Do RF Drone Benchmarks Overstate? A Controlled Study and Theory of Data Leakage in UAV Signal Identification

• URL: http://arxiv.org/abs/2607.01025v1
• Date: Wed, 01 Jul 2026 14:57:44 GMT
• Title: How Much Do RF Drone Benchmarks Overstate? A Controlled Study and Theory of Data Leakage in UAV Signal Identification
• Abstract summary: Radio-frequency (RF) sensing is a central modality for counter-unmanned-aerial-system (counter-UAS) defence.<n>Reported accuracies for RF-based drone detection and identification are often very high.<n>Many are obtained using cross-validation that splits a small number of continuous recordings into short segments.<n>This can place near-duplicate slices of the same recording in both training and test partitions, creating data leakage.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Radio-frequency (RF) sensing is a central modality for counter-unmanned-aerial-system (counter-UAS) defence because it exploits the control, telemetry, and video links between a drone and its operator. Reported accuracies for RF-based drone detection and identification are often very high, but many are obtained using cross-validation that splits a small number of continuous recordings into short segments. This can place near-duplicate slices of the same recording in both training and test partitions, creating data leakage. We study this leakage problem through theory and measurement. We formalise the optimism of segment-level cross-validation and show, using Cover's function-counting theorem, that a classifier can exactly memorise the recording-to-label map when the number of independent recordings, R, is small relative to the feature dimension, d. In particular, this can occur when 2R is less than or approximately equal to d. Under these conditions, naive accuracy approaches 1, and the inflation gap approaches 1 - ACC*, where ACC* is the Bayes accuracy. The inflation eases only once R grows beyond this separability threshold. A controlled synthetic experiment with 10 seeds confirms the predicted curves: naive balanced accuracy rises from the Bayes level toward 1.0 as recording-specific nuisance variation grows, while honest recording-grouped evaluation declines to chance, with a gap reaching about 0.5. On the public DroneRF dataset, pooled leave-one-recording-out cross-validation shows drone type identification, AR versus Bebop, collapsing from a naive macro-F1 of 0.74 to 0.46, the two-class chance level. A leakage-pathway ablation attributes essentially all of the inflation to segment-level leakage.

Related papers

• Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection [66.44527094471619]
  We present RECAL, an unsupervised framework using relation-balanced masked graph learning to better capture rare interaction patterns.<n>On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset by 0.88, 0.82, and 0.42 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-09-15T00:31:01Z)
• Rigid-Covert GNSS Spoofing of UAV Swarms: A Structural Blind Spot, Its Detection Limit, and Absolute-Anchor Defenses [0.0]
  Cooperative UAV-s defenses commonly cross-check positions against measured interdrone geometry.<n>We show that this relative-geometry channel has a structural blind spot: a common, slowly varying translation (a rigidcovert shift, RigidShift) preserves all pairwise distances and is therefore unobservable to any relativewarm-only detector.<n>We present a centralized anchor-rooted recovery pipeline that reconstructs swarm geometry from inter-drone ranges.
  arXiv  Detail & Related papers  (2026-08-07T07:15:55Z)
• A global predicted-fMRI drive signal from TRIBE does not predict YouTube replay heatmaps [0.0]
  multimodal brain-encoding models predict fMRI responses to naturalistic video with high accuracy.<n>We run TRIBE on 48 YouTube videos and reduce its predicted cortical response to a per-second engagement curve.<n>Running the probe on TRIBE's input streams reveals at most a small, borderline visual-stream signal.<n>We release code, a video-ID manifest, and a heatmap-acquisition method robust to YouTube's SABR streaming.
  arXiv  Detail & Related papers  (2026-07-01T19:03:17Z)
• Real-Time Source-Free Object Detection [40.10658212485104]
  Real-world detectors for autonomous driving, surveillance, and robotics must handle domain-shifts under strict latency and memory constraints.<n>We show that building on YOLOv10, an NMS-free dual-head detector, we achieve state-of-the-art adaptation accuracy while being faster and more compact.
  arXiv  Detail & Related papers  (2026-06-30T15:38:56Z)
• Anticipating the Optimism Gap: Predicting Distribution-Shift Degradation of RF-Impairment Detectors from In-Distribution Statistics [0.0]
  Detectors for radio-frequency impairments are usually reported with a single AUC measured on the distribution they were tuned on.<n>We ask whether this optimism can be predicted before any out-of-distribution data is seen.
  arXiv  Detail & Related papers  (2026-06-20T14:10:59Z)
• Beyond ROC-AUC: Operating-Point Performance Reporting for Biometric Verification [0.0]
  A biometric verifier is often deployed with a strict false match budget, so only a narrow, low false match rate (FMR) slice of the score range is used.<n>In practice, a single area under the receiver operating characteristic curve (ROC-AUC) is still reported as the resolution.<n>A system that looks stronger on full ROC-AUC is shown to be significantly worse at FMR = 10-3.
  arXiv  Detail & Related papers  (2026-06-13T15:28:54Z)
• Counterfactual Likelihood Tests for Indirect Influence in Private Reasoning Channels [51.56484100374058]
  This paper presents a counterfactual likelihood test for measuring influence between private reasoning channels.<n>The method replaces an upstream private block with a length-matched donor block, holds the public token sequence and downstream target fixed, and measures the downstream target's negative-log-likelihood shift.
  arXiv  Detail & Related papers  (2026-05-18T20:27:43Z)
• Beyond the Fold: Quantifying Split-Level Noise and the Case for Leave-One-Dataset-Out AU Evaluation [1.7159330469510958]
  We show that cross-validation itself introduces measurable variance.<n>On BP4D+, repeated 3-fold subject-exclusive splits produce an empirical noise floor of $pm 0.065$ in average F1.<n>We further evaluate cross-dataset robustness using a Leave-One-Dataset-Out protocol across five AU datasets.
  arXiv  Detail & Related papers  (2026-04-02T15:28:55Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• TARG: Training-Free Adaptive Retrieval Gating for Efficient RAG [46.122203287541005]
  Training-free Adaptive Retrieval Gating (TARG) is a single-shot policy that decides when to retrieve using only a short, no-latency draft from the base model.<n>On NQ-Open, TriviaQA, and PopQA, TARG consistently shifts the accuracy-efficiency frontier.
  arXiv  Detail & Related papers  (2025-11-12T23:09:52Z)
• Federated Spatiotemporal Graph Learning for Passive Attack Detection in Smart Grids [2.721477719641864]
  This paper introduces a graph-centric, multimodal detector that fuses physical-layer and behavioral indicators over temporal windows to detect passive attacks.<n>The model achieves a testing accuracy of 98.32% per-timestep and 93.35% per-sequence at 0.15% FPR.
  arXiv  Detail & Related papers  (2025-09-29T08:52:30Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Imbalanced Aircraft Data Anomaly Detection [103.01418862972564]
  Anomaly detection in temporal data from sensors under aviation scenarios is a practical but challenging task. We propose a Graphical Temporal Data Analysis framework. It consists three modules, named Series-to-Image (S2I), Cluster-based Resampling Approach using Euclidean Distance (CRD) and Variance-Based Loss (VBL)
  arXiv  Detail & Related papers  (2023-05-17T09:37:07Z)
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times [51.61278695776151]
  Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively. Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
  arXiv  Detail & Related papers  (2021-01-11T19:43:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.