What the Detector Can See: Evaluating CPS Anomaly Detectors Independently of the Decision Rule

• URL: http://arxiv.org/abs/2608.02821v1
• Date: Mon, 03 Aug 2026 19:35:00 GMT
• Title: What the Detector Can See: Evaluating CPS Anomaly Detectors Independently of the Decision Rule
• Abstract summary: Anomaly detectors are often the last line of defense for cyber-physical systems.<n>These detectors are usually compared using precision, recall, or F1 at a single operating point.<n>We treat a CPS anomaly detector as a two-stage pipeline: Stage 1 maps observations to residuals, and Stage 2 maps residuals to alarms.<n>We evaluate five detectors across three CPS benchmarks: SWaT, WADI, and HAI.
• Score: 6.145872664954402
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Anomaly detectors are often the last line of defense for cyber-physical systems (CPS). But detectors built in very different ways, from deep neural networks to invariant templates, are usually compared using precision, recall, or F1 at a single operating point. These scores mix two separate things: how well the detector represents the physical process, and how well its alarm threshold is set. We therefore treat a CPS anomaly detector as a two-stage pipeline: Stage 1 maps observations to residuals, and Stage 2 maps residuals to alarms. Instead of scoring only the final alarms, we evaluate Stage 1 directly using normalized residual energy, which has an exact connection to the Kullback-Leibler divergence from the trained-normal reference distribution. Because it does not depend on a specific alarm rule, it can separately measure attack separation, stability across the train-test gap, and the compactness with which a detector encodes the plant. Without any per-detector tuning, we apply this evaluation to five detectors -- GDN, FuSAGNet, TranAD, NSIBF, and GeCo -- across three CPS benchmarks: SWaT, WADI, and HAI. Although the detectors have similar ROC-AUC values on SWaT, their performance differs by more than an order of magnitude at a common false-alarm rate. Rankings also change across testbeds: TranAD ranks first on HAI but last on SWaT, while NSIBF ranks first on WADI but last on HAI. On WADI, localized attacks can evade detectors that pool evidence across all channels, helping explain why NSIBF outperforms methods that do well on other benchmarks. These results show that detection failure can come from different sources: a weak representation, poor threshold calibration, or an attack with little physical effect. A decision-rule-free analysis helps separate these causes.

Related papers

• Differentiable Interval Bottlenecks for Interpretable Anomaly Detection in Numerical Data [0.5213778368155993]
  DIFFINT is an autoencoder whose latent bottleneck is structured as a set of soft, axis-aligned interval memberships.<n>It is the only interpretable detector in the statistically-tied leading cluster of seven methods.
  arXiv  Detail & Related papers  (2026-09-03T14:05:27Z)
• Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer [36.02174685737611]
  We find that detector rankings reverse across deployments, every detector exceeds $80%$ FPR95 on at least one domain, and the preferred detector depends on both the in-distribution data and the underlying VLM.<n>We introduce the Complementary Evidence Guard (CEG), a detector-agnostic wrapper that preserves complementary evidence through a non-compensatory fusion of the base detector, level, and sharpness.
  arXiv  Detail & Related papers  (2026-07-29T08:00:49Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Token-Level Generalization in LoRA Adapter Backdoors: Attack Characterization and Behavioral Detection [0.0]
  We show that LoRA can be reliably backdoored through training data poisoning.<n>The attack scales monotonically with rank, and the chosen trigger-anchor token is both trigger-dependent and base-model-dependent.<n> Behavioral detection is the operationally portable result for supply chain scanning.
  arXiv  Detail & Related papers  (2026-05-28T16:32:25Z)
• Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text [0.039998518782208774]
  We show that the token-level signal distinguishing human and machine text is non-uniform across the hidden space of the detector model.<n>We introduce a learned local calibration step grounded in Bayesian decision theory.<n>This single intervention dramatically and consistently improves detection performance across all baseline detectors and all datasets we consider.
  arXiv  Detail & Related papers  (2026-05-07T13:59:07Z)
• Knowledge-Guided Failure Prediction: Detecting When Object Detectors Miss Safety-Critical Objects [0.764671395172401]
  A representation-based monitoring framework treats missed safety-critical detections as anomalies to be detected at runtime.<n>A key property is that when either the detector is operating outside its competence or the visual foundation model itself encounters novel inputs, the two embeddings diverge.<n>On COCO person detection, applying KGFP as a selective-prediction gate raises person recall among accepted images from 64.3% to 84.5% at 5% False Positive Rate (FPR)<n>Our code, models, and features are published at https://gitlab.cc-asp.fraunhofer.de/iosb_public/KGFP.
  arXiv  Detail & Related papers  (2026-03-26T14:38:20Z)
• ECoLAD: Deployment-Oriented Evaluation for Automotive Time-Series Anomaly Detection [1.7751300245073598]
  Time-series anomaly detectors are commonly compared on workstation-class hardware under unconstrained execution.<n>In-vehicle monitoring, however, requires predictable latency and stable behavior under limited CPU parallelism.<n>We present ECoLAD, a deployment-oriented evaluation protocol instantiated as an empirical study on proprietary automotive telemetry.
  arXiv  Detail & Related papers  (2026-03-11T16:08:56Z)
• Mitigating the Mutual Error Amplification for Semi-Supervised Object Detection [92.52505195585925]
  We propose a Cross Teaching (CT) method, aiming to mitigate the mutual error amplification by introducing a rectification mechanism of pseudo labels. In contrast to existing mutual teaching methods that directly treat predictions from other detectors as pseudo labels, we propose the Label Rectification Module (LRM)
  arXiv  Detail & Related papers  (2022-01-26T03:34:57Z)
• DETReg: Unsupervised Pretraining with Region Priors for Object Detection [103.93533951746612]
  DETReg is a new self-supervised method that pretrains the entire object detection network. During pretraining, DETReg predicts object localizations to match the localizations from an unsupervised region proposal generator. It simultaneously aligns the corresponding feature embeddings with embeddings from a self-supervised image encoder.
  arXiv  Detail & Related papers  (2021-06-08T17:39:14Z)
• DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows [52.31831255787147]
  We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA) Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution. Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
  arXiv  Detail & Related papers  (2021-05-30T22:07:13Z)
• Probabilistic Ranking-Aware Ensembles for Enhanced Object Detections [50.096540945099704]
  We propose a novel ensemble called the Probabilistic Ranking Aware Ensemble (PRAE) that refines the confidence of bounding boxes from detectors. We also introduce a bandit approach to address the confidence imbalance problem caused by the need to deal with different numbers of boxes.
  arXiv  Detail & Related papers  (2021-05-07T09:37:06Z)
• Probabilistic two-stage detection [83.9604523643406]
  We show how to build a probabilistic two-stage detector from any state-of-the-art one-stage detector. The resulting detectors are faster and more accurate than both their one- and two-stage precursors.
  arXiv  Detail & Related papers  (2021-03-12T18:56:17Z)
• Object Detection Made Simpler by Eliminating Heuristic NMS [70.93004137521946]
  We show a simple NMS-free, end-to-end object detection framework. We attain on par or even improved detection accuracy compared with the original one-stage detector.
  arXiv  Detail & Related papers  (2021-01-28T02:38:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.