A Novel Evaluation Metric for Unsupervised Learning in AIS-Based Maritime Anomaly Detection: MADQI

• URL: http://arxiv.org/abs/2605.30388v1
• Date: Thu, 28 May 2026 11:02:41 GMT
• Title: A Novel Evaluation Metric for Unsupervised Learning in AIS-Based Maritime Anomaly Detection: MADQI
• Abstract summary: This paper introduces a new systematic framework for detecting anomalies in maritime Automatic Identification System (AIS) datasets.<n>The proposed framework integrates four interconnected metrics: Anomaly Rate Consistency (ARC), Physical Plausibility Score (PPS), Score Distribution Separation (SDS) and Extreme Case Evidence (ECE)<n> Experimental results on the AIS dataset show that the proposed framework achieved a MADQI score of 80.37%, demonstrating its effectiveness for unsupervised anomaly detection.
• Score: 0.5424799109837065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces a new systematic framework for detecting anomalies in maritime Automatic Identification System (AIS) datasets. These anomalies include abnormal vessel behaviours related to speed, position jumps, time gaps, and turn angles. Although unsupervised learning algorithms such as Isolation Forest are widely used for detecting anomalous vessel movements, they often lack systematic and meaningful evaluation measures. To address this limitation, we propose a novel quality metric called Maritime Anomaly Detection Quality Index (MADQI). The prosed MADQI is a composite index designed to evaluate the anomaly detection performance of machine learning models without requiring labelled data. The proposed framework uses Haversine distance calculations to analyse AIS datasets and identify anomalies based on their spatial and behavioural characteristics. The proposed MADQI evaluation framework integrates four interconnected metrics: Anomaly Rate Consistency (ARC), Physical Plausibility Score (PPS), Score Distribution Separation (SDS), and Extreme Case Evidence (ECE). These metrics are combined through automatic normalisation using multi-chunk evaluation and adaptive scaling techniques. Experimental results on the AIS dataset show that the proposed framework achieved a MADQI score of 80.37%, demonstrating its effectiveness for unsupervised anomaly detection. In particular, the algorithm performed strongly in identifying abnormal vessel behaviour. Among the individual MADQI components, ECE and ARC achieved scores of 0.907 and 1.000, respectively, indicating excellent capability in detecting extreme anomalies and maintaining anomaly rate consistency. Overall, these results are encouraging and demonstrate that the proposed framework provides a reliable and meaningful approach for evaluating unsupervised anomaly detection in maritime AIS data.

Related papers

• OOD-RL-Bench: A Benchmark Framework for Out-of-Distribution Detection in Reinforcement Learning [4.173831365316023]
  We present OOD-RL-Bench, a framework designed to evaluate OOD detectors against categories of anomalies injected into trajectories.<n> Detectors and anomaly injectors are integrated through shared interfaces and configuration.<n>We assess the performance of each detector across a suite of anomaly types using matched-time AUROC, matched-time AUPRC, matched-time false-positive rate, detection delay, and segmented-onset metrics.
  arXiv  Detail & Related papers  (2026-07-14T08:57:04Z)
• Redefining Maritime Anomaly Detection via Equation-Grounded Synthetic Anomalies [11.891479226134651]
  Maritime anomaly detection is essential for ensuring maritime safety, security, and efficient traffic management at sea.<n>Most publicly available AIS datasets lack predefined anomaly labels.<n>We introduce a unified score-synthesize-label pipeline that produces LLM-guided plausibility scores, uses them to synthesize anomalies, and assigns timestamp-level labels.
  arXiv  Detail & Related papers  (2026-06-29T02:56:46Z)
• BenchReAD: A systematic benchmark for retinal anomaly detection [23.15668882564837]
  We introduce a benchmark for retinal anomaly detection, which is comprehensive and systematic in terms of data and algorithm.<n>We find that a fully supervised approach leveraging disentangled representations of abnormalities (DRA) achieves the best performance but suffers from significant drops in performance when encountering certain unseen anomalies.<n>Inspired by the memory bank mechanisms in one-class supervised learning, we propose NFM-DRA, which integrates DRA with a Normal Feature Memory to mitigate the performance degradation.
  arXiv  Detail & Related papers  (2025-07-14T17:13:08Z)
• Detect, Classify, Act: Categorizing Industrial Anomalies with Multi-Modal Large Language Models [27.008700759998945]
  We propose VELM, a novel pipeline for anomaly classification.<n>We introduce MVTec-AC and VisA-AC, refined versions of the widely used MVTec-AD and VisA-AC datasets.<n>Our approach achieves a state-of-the-art anomaly classification accuracy of 80.4% on MVTec-AD, exceeding the prior baselines by 5%, and 84% on MVTec-AC, demonstrating the effectiveness of VELM.
  arXiv  Detail & Related papers  (2025-05-05T13:08:25Z)
• Adaptive Signal Analysis for Automated Subsurface Defect Detection Using Impact Echo in Concrete Slabs [0.0]
  This pilot study presents a novel, automated, and scalable methodology for detecting subsurface defect-prone regions in concrete slabs.<n>The approach integrates advanced signal processing, clustering, and visual analytics to identify subsurface anomalies.<n>The results demonstrate the robustness of the methodology, consistently identifying defect-prone areas with minimal false positives and few missed defects.
  arXiv  Detail & Related papers  (2024-12-23T20:05:53Z)
• MeLIAD: Interpretable Few-Shot Anomaly Detection with Metric Learning and Entropy-based Scoring [2.394081903745099]
  We propose MeLIAD, a novel methodology for interpretable anomaly detection. MeLIAD is based on metric learning and achieves interpretability by design without relying on any prior distribution assumptions of true anomalies. Experiments on five public benchmark datasets, including quantitative and qualitative evaluation of interpretability, demonstrate that MeLIAD achieves improved anomaly detection and localization performance.
  arXiv  Detail & Related papers  (2024-09-20T16:01:43Z)
• Performance Metric for Multiple Anomaly Score Distributions with Discrete Severity Levels [4.66313002591741]
  We propose a weighted sum of the area under the receiver operating characteristic curve (WS-AUROC) for classifying severity levels based on anomaly scores. We also propose an anomaly detector that achieves clear separation of distributions and outperforms the ablation models on the WS-AUROC and AUROC metrics.
  arXiv  Detail & Related papers  (2024-08-09T02:17:49Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [89.92916473403108]
  This paper proposes a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods.<n>The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics.<n>We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• MSFlow: Multi-Scale Flow-based Framework for Unsupervised Anomaly Detection [124.52227588930543]
  Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications. An inconspicuous yet powerful statistics model, the normalizing flows, is appropriate for anomaly detection and localization in an unsupervised fashion. We propose a novel Multi-Scale Flow-based framework dubbed MSFlow composed of asymmetrical parallel flows followed by a fusion flow. Our MSFlow achieves a new state-of-the-art with a detection AUORC score of up to 99.7%, localization AUCROC score of 98.8%, and PRO score of 97.1%.
  arXiv  Detail & Related papers  (2023-08-29T13:38:35Z)
• Cloud Failure Prediction with Hierarchical Temporary Memory: An Empirical Assessment [64.73243241568555]
  Hierarchical Temporary Memory (HTM) is an unsupervised learning algorithm inspired by the features of the neocortex. This paper presents the first systematic study that assesses HTM in the context of failure prediction.
  arXiv  Detail & Related papers  (2021-10-06T07:09:45Z)
• TELESTO: A Graph Neural Network Model for Anomaly Classification in Cloud Services [77.454688257702]
  Machine learning (ML) and artificial intelligence (AI) are applied on IT system operation and maintenance. One direction aims at the recognition of re-occurring anomaly types to enable remediation automation. We propose a method that is invariant to dimensionality changes of given data.
  arXiv  Detail & Related papers  (2021-02-25T14:24:49Z)
• NADS: Neural Architecture Distribution Search for Uncertainty Awareness [79.18710225716791]
  Machine learning (ML) systems often encounter Out-of-Distribution (OoD) errors when dealing with testing data coming from a distribution different from training data. Existing OoD detection approaches are prone to errors and even sometimes assign higher likelihoods to OoD samples. We propose Neural Architecture Distribution Search (NADS) to identify common building blocks among all uncertainty-aware architectures.
  arXiv  Detail & Related papers  (2020-06-11T17:39:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.