Monitoring Robustness and Individual Fairness

• URL: http://arxiv.org/abs/2506.00496v1
• Date: Sat, 31 May 2025 10:27:54 GMT
• Title: Monitoring Robustness and Individual Fairness
• Abstract summary: We propose runtime monitoring of input-output robustness of deployed, black-box AI models.<n>We show that the monitoring problem can be cast as the fixed-radius nearest neighbor (FRNN) search problem.<n>We present our tool Clemont, which offers a number of lightweight monitors.
• Score: 7.922558880545528
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Input-output robustness appears in various different forms in the literature, such as robustness of AI models to adversarial or semantic perturbations and individual fairness of AI models that make decisions about humans. We propose runtime monitoring of input-output robustness of deployed, black-box AI models, where the goal is to design monitors that would observe one long execution sequence of the model, and would raise an alarm whenever it is detected that two similar inputs from the past led to dissimilar outputs. This way, monitoring will complement existing offline ``robustification'' approaches to increase the trustworthiness of AI decision-makers. We show that the monitoring problem can be cast as the fixed-radius nearest neighbor (FRNN) search problem, which, despite being well-studied, lacks suitable online solutions. We present our tool Clemont, which offers a number of lightweight monitors, some of which use upgraded online variants of existing FRNN algorithms, and one uses a novel algorithm based on binary decision diagrams -- a data-structure commonly used in software and hardware verification. We have also developed an efficient parallelization technique that can substantially cut down the computation time of monitors for which the distance between input-output pairs is measured using the $L_\infty$ norm. Using standard benchmarks from the literature of adversarial and semantic robustness and individual fairness, we perform a comparative study of different monitors in \tool, and demonstrate their effectiveness in correctly detecting robustness violations at runtime.

Related papers

• $D^2$-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation-Aware Routing [63.49501120848927]
  Safety monitoring for large language models (D-LLMs) remains largely unexplored.<n>We propose $D2$-Monitor, a bi-level safety monitor for D-LLMs.<n>$D2$-Monitor adopts a lightweight probe as an always-on monitor to jointly estimate hesitation and perform base classification.
  arXiv  Detail & Related papers  (2026-05-25T14:22:21Z)
• SMFormer: Empowering Self-supervised Stereo Matching via Foundation Models and Data Augmentation [55.67206878777881]
  We propose SMFormer, a framework integrating more reliable self-supervision guided by the Vision Foundation Model (VFM) and data augmentation.<n> SMFormer achieves state-of-the-art (SOTA) performance among self-supervised methods and even competes on par with supervised ones.
  arXiv  Detail & Related papers  (2026-04-11T13:56:41Z)
• Gated Condition Injection without Multimodal Attention: Towards Controllable Linear-Attention Transformers [69.60693631859411]
  This paper explores controllable diffusion models built upon linear attention architectures.<n>We propose a novel controllable diffusion framework tailored for linear attention backbones like SANA.<n>Our approach achieves state-of-the-art controllable generation performance based on linear-attention models.
  arXiv  Detail & Related papers  (2026-03-29T12:31:34Z)
• Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory [11.144603446849674]
  Chain-of-thought (CoT) monitors are systems that analyze reasoning traces to detect when outputs may exhibit attributes of interest.<n>In this paper, we show that non-zero mutual information between CoT and output is a necessary but not sufficient condition for CoT monitorability.
  arXiv  Detail & Related papers  (2026-02-20T15:50:30Z)
• MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detection [28.5803063507761]
  Video Anomaly Detection (VAD) aims to locate unusual activities or behaviors within videos.<n>Online VAD has seldom received attention due to real-time constraints and computational intensity.<n>We introduce a novel Memory-based online scoring queue scheme for Training-free VAD (MoniTor)
  arXiv  Detail & Related papers  (2025-10-24T13:28:29Z)
• Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking [54.43083499412643]
  Test-time algorithms that combine the generative power of language models with process verifiers offer a promising lever for eliciting new reasoning capabilities.<n>We introduce a new process-guided test-time sampling algorithm, VGB, which uses theoretically grounded backtracking to achieve provably better robustness to verifier errors.
  arXiv  Detail & Related papers  (2025-10-03T16:21:14Z)
• Monitoring of Static Fairness [5.866925594233267]
  We present a general framework of runtime verification of algorithmic fairness.<n>We introduce a specification language that can model many common algorithmic fairness properties.<n>We show how we can monitor if a bank is fair in giving loans to applicants from different social backgrounds.
  arXiv  Detail & Related papers  (2025-07-03T10:20:04Z)
• Reinforcing Multimodal Understanding and Generation with Dual Self-rewards [56.08202047680044]
  Large language models (LLMs) unify cross-model understanding and generation into a single framework.<n>Current solutions require external supervision (e.g., human feedback or reward models) and only address unidirectional tasks.<n>We introduce a self-supervised dual reward mechanism to reinforce the understanding and generation capabilities of LMMs.
  arXiv  Detail & Related papers  (2025-06-09T17:38:45Z)
• SafetyNet: Detecting Harmful Outputs in LLMs by Modeling and Monitoring Deceptive Behaviors [2.07180164747172]
  High-risk industries like nuclear and aviation use real-time monitoring to detect dangerous system conditions.<n>We propose a real-time framework to predict harmful AI outputs before they occur by using an unsupervised approach.
  arXiv  Detail & Related papers  (2025-05-20T12:49:58Z)
• Benchmarking Vision Foundation Models for Input Monitoring in Autonomous Driving [7.064497253920508]
  Vision Foundation Models (VFMs) as feature extractors and density modeling techniques are proposed.<n>A comparison with state-of-the-art binary OOD classification methods reveals that VFM embeddings with density estimation outperform existing approaches in identifying OOD inputs.<n>Our method detects high-risk inputs likely to cause errors in downstream tasks, thereby improving overall performance.
  arXiv  Detail & Related papers  (2025-01-14T12:51:34Z)
• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• Monitizer: Automating Design and Evaluation of Neural Network Monitors [0.9236074230806581]
  The behavior of neural networks (NNs) on previously unseen types of data (out-of-distribution or OOD) is typically unpredictable. This can be dangerous if the network's output is used for decision-making in a safety-critical system. We present a tool for users and developers of NN monitors.
  arXiv  Detail & Related papers  (2024-05-16T13:19:51Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens [6.329470650220206]
  The aim of this work is to highlight the relatively under-appreciated complexity of designing a monitoring strategy. We consider an ML-based risk prediction algorithm for predicting unplanned readmissions. Results from this case study emphasize the seemingly simple (and obvious) fact that not all monitoring systems are created equal.
  arXiv  Detail & Related papers  (2023-11-20T00:15:16Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing [88.35145788575348]
  Image anomaly detection (IAD) is an emerging and vital computer vision task in industrial manufacturing. The lack of a uniform IM benchmark is hindering the development and usage of IAD methods in real-world applications. We construct a comprehensive image anomaly detection benchmark (IM-IAD), which includes 19 algorithms on seven major datasets.
  arXiv  Detail & Related papers  (2023-01-31T01:24:45Z)
• A Bayesian Detect to Track System for Robust Visual Object Tracking and Semi-Supervised Model Learning [1.7268829007643391]
  We ad-dress problems in a Bayesian tracking and detection framework parameterized by neural network outputs. We propose a particle filter-based approximate sampling algorithm for tracking object state estimation. Based on our particle filter inference algorithm, a semi-supervised learn-ing algorithm is utilized for learning tracking network on intermittent labeled frames.
  arXiv  Detail & Related papers  (2022-05-05T00:18:57Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows [52.31831255787147]
  We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA) Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution. Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
  arXiv  Detail & Related papers  (2021-05-30T22:07:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.