A Standardized Framework for Machine Learning in Power System Protection

• URL: http://arxiv.org/abs/2608.20181v1
• Date: Thu, 20 Aug 2026 15:35:36 GMT
• Title: A Standardized Framework for Machine Learning in Power System Protection
• Abstract summary: This paper proposes a standardization-oriented framework that treats evaluation design as part of the scientific contribution.<n>It defines seven required study dimensions: protection objective, physical scope, observability, timing and decision windows, targets and sample validity, validation protocol, and evaluation outputs.
• Score: 8.877321413840214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Studies of machine-learning-based power-system protection increasingly report near-perfect scores, yet the meaning of those scores depends strongly on the evaluation setting. Protection task, physical scope, measurements, timing, targets, preprocessing, and validation often vary jointly and remain incompletely specified. This paper proposes a standardization-oriented framework that treats evaluation design as part of the scientific contribution. It defines seven required study dimensions: protection objective, physical scope, observability, timing and decision windows, targets and sample validity, validation protocol, and evaluation outputs. The framework is instantiated in a bounded case study on the public PROTECT-90 electromagnetic-transient benchmark, comprising 9022 simulated episodes from a 90 kV double-line topology, for onset-conditioned fault classification and localization. Under centralized sensing, simulation-metadata-aligned 20 ms windows, and episode-grouped validation, a multi-layer perceptron (MLP) achieved a five-fold mean macro-averaged F1 score of 0.991 +/- 0.001 for classification and a localization mean absolute error of 10.20 +/- 0.25% of line length (mean +/- std across episode-grouped folds). Extending the decision horizon to 50 ms preserved this task-dependent performance asymmetry, while reduced observability approximately doubled the MLP localization error but had little effect on classification. A synchronized two-ended conventional locator outperformed the learning locators under its richer clean information set, and measurement degradation showed that clean predictive performance did not determine robustness. The framework turns evaluation assumptions into explicit, reproducible evidence and provides a basis for more comparable, auditable evaluation and future certification-oriented assessment of machine-learning protection functions.

Related papers

• Beyond Local Accuracy: A Protocol-Level Identifiability Audit for Controlled LLM Reasoning Evaluation [6.047519836191728]
  We formalize a protocol-level identifiability audit over a finite behavioral policy class.<n>The audit requires zero model calls and resolves our diagnostic case.<n>This case shows how evaluation-design validity can be checked structurally before model inference.
  arXiv  Detail & Related papers  (2026-08-13T14:49:47Z)
• UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation [0.0]
  UpliftBench evaluates 12 uplift estimators under an outer-test-isolated, multi-objective protocol.<n>We show the disagreement is substantially about metrics, not models.
  arXiv  Detail & Related papers  (2026-08-02T00:51:44Z)
• From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape [79.30826980815927]
  rubrics are explicit criteria sets that transform complex quality judgments into structured and actionable standards.<n>We systematically organize existing rubric designs, examine their construction and optimization, and analyze their role across evaluation and training.
  arXiv  Detail & Related papers  (2026-06-07T13:34:55Z)
• From Frames to Events: Rethinking Evaluation in Human-Centric Video Anomaly Detection [9.404376027901277]
  Pose-based Video Anomaly Detection (VAD) has gained significant attention for its privacy-preserving nature and robustness to environmental variations.<n>Traditional frame-level evaluations treat video as a collection of isolated frames, fundamentally misaligned with how anomalies manifest and are acted upon in the real world.<n>We introduce two strategies for temporal event localization: a score-refinement pipeline with hierarchical Gaussian smoothing and adaptive binarization, and an end-to-end Dual-Branch Model that directly generates event-level detections.
  arXiv  Detail & Related papers  (2026-04-10T13:52:18Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training [59.493415006017635]
  Pre-trained Multi-modal Large Language Models (MLLMs) provide a knowledge-rich foundation for post-training.<n>Current evaluation relies on testing after supervised fine-tuning, which introduces laborious additional training and autoregressive decoding costs.<n>We propose RADAR, an efficient ability-centric evaluation framework for Revealing Asymmetric Development of Abilities in MLLM pRe-training.
  arXiv  Detail & Related papers  (2026-02-13T12:56:31Z)
• Benchmarking Corruption Robustness of LVLMs: A Discriminative Benchmark and Robustness Alignment Metric [49.393713730706445]
  We introduce Bench-C, a benchmark emphasizing discriminative samples for assessing corruption robustness.<n>We propose the Robustness Alignment Score (RAS), a unified metric that measures degradation in logit-level prediction structure.
  arXiv  Detail & Related papers  (2025-11-24T12:07:56Z)
• Evaluating the Ability of Large Language Models to Identify Adherence to CONSORT Reporting Guidelines in Randomized Controlled Trials: A Methodological Evaluation Study [7.142913983218931]
  This study aimed to evaluate the accuracy and reliability of contemporary LLMs in identifying adherence of published RCTs.<n>We constructed a golden standard dataset of 150 published RCTs spanning diverse medical specialties.<n>Top-performing models, Gemini-2.5-Flash and DeepSeek-R1, achieved nearly identical macro F1 scores of 0.634 and Cohen's Kappa coefficients of 0.280 and 0.282, respectively.
  arXiv  Detail & Related papers  (2025-11-17T08:05:15Z)
• Benchmarking Machine Learning Models for Fault Classification and Localization in Power System Protection [10.403249318465468]
  This work presents a comparative benchmarking study of classical machine learning models for fault classification (FC) and fault localization (FL) in power system protection based on EMT data.<n>The best-performing FC model achieved an F1 score of 0.992$pm$0.001, while the top FL model reached an R2 of 0.806$pm$0.008 with a mean processing time of 0.563 ms.
  arXiv  Detail & Related papers  (2025-10-01T12:44:14Z)
• Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processes [65.91730154730905]
  In applications of offline reinforcement learning to observational data, such as in healthcare or education, a general concern is that observed actions might be affected by unobserved factors. Here we tackle this by considering off-policy evaluation in a partially observed Markov decision process (POMDP) We extend the framework of proximal causal inference to our POMDP setting, providing a variety of settings where identification is made possible.
  arXiv  Detail & Related papers  (2021-10-28T17:46:14Z)
• Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection [85.53263670166304]
  One-stage detector basically formulates object detection as dense classification and localization. Recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization.
  arXiv  Detail & Related papers  (2020-06-08T07:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.