Hoi2Anomaly: An Explainable Anomaly Detection Approach Guided by Human-Object Interaction

• URL: http://arxiv.org/abs/2503.10508v2
• Date: Sat, 15 Mar 2025 05:44:22 GMT
• Title: Hoi2Anomaly: An Explainable Anomaly Detection Approach Guided by Human-Object Interaction
• Authors: Yuhan Wang, Cheng Liu, Daou Zhang, Weichao Wu,
• Abstract summary: We propose a novel approach to anomaly detection, termed Hoi2Anomaly, which aims to achieve precise discrimination and localization of anomalies.<n>The proposed methodology involves the construction of a multi-modal instruction tuning dataset comprising human-object interaction (HOI) pairs in anomalous scenarios.<n>The experimental results demonstrate that Hoi2Anomaly surpasses existing generative approaches in terms of precision and explainability.
• Score: 4.504505809016945
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the domain of Image Anomaly Detection (IAD), Existing methods frequently exhibit a paucity of fine-grained, interpretable semantic information, resulting in the detection of anomalous entities or activities that are susceptible to machine illusions. This deficiency often leads to the detection of anomalous entities or actions that are susceptible to machine illusions and lack sufficient explanation. In this thesis, we propose a novel approach to anomaly detection, termed Hoi2Anomaly, which aims to achieve precise discrimination and localization of anomalies. The proposed methodology involves the construction of a multi-modal instruction tuning dataset comprising human-object interaction (HOI) pairs in anomalous scenarios. Second, we have trained an HOI extractor in threat scenarios to localize and match anomalous actions and entities. Finally, explanatory content is generated for the detected anomalous HOI by fine-tuning the visual language pretraining (VLP) framework. The experimental results demonstrate that Hoi2Anomaly surpasses existing generative approaches in terms of precision and explainability. We will release Hoi2Anomaly for the advancement of the field of anomaly detection.

Related papers

• Hybrid LLM-Enhanced Intrusion Detection for Zero-Day Threats in IoT Networks [6.087274577167399]
  This paper presents a novel approach to intrusion detection by integrating traditional signature-based methods with the contextual understanding capabilities of the GPT-2 Large Language Model (LLM)<n>We propose a hybrid IDS framework that merges the robustness of signature-based techniques with the adaptability of GPT-2-driven semantic analysis.<n> Experimental evaluations on a representative intrusion dataset demonstrate that our model enhances detection accuracy by 6.3%, reduces false positives by 9.0%, and maintains near real-time responsiveness.
  arXiv  Detail & Related papers  (2025-07-10T04:10:03Z)
• Surfacing Semantic Orthogonality Across Model Safety Benchmarks: A Multi-Dimensional Analysis [4.3659097510044855]
  Our evaluation reveals distinct semantic clusters using UMAP dimensionality reduction and kmeans clustering.<n>We identify six primary harm categories with varying benchmark representation.
  arXiv  Detail & Related papers  (2025-05-23T08:53:11Z)
• Lie Detector: Unified Backdoor Detection via Cross-Examination Framework [68.45399098884364]
  We propose a unified backdoor detection framework in the semi-honest setting.<n>Our method achieves superior detection performance, improving accuracy by 5.4%, 1.6%, and 11.9% over SoTA baselines.<n> Notably, it is the first to effectively detect backdoors in multimodal large language models.
  arXiv  Detail & Related papers  (2025-03-21T06:12:06Z)
• Enhanced Online Grooming Detection Employing Context Determination and Message-Level Analysis [2.424910201171407]
  Online grooming (OG) is a prevalent threat facing predominately children online, with groomers using deceptive methods to prey on the vulnerability of children on social media/messaging platforms. Existing solutions focus on the signature analysis of child abuse media, which does not effectively address real-time OG detection. This paper proposes that OG attacks are complex, requiring the identification of specific communication patterns between adults and children.
  arXiv  Detail & Related papers  (2024-09-12T11:37:34Z)
• A practical approach to evaluating the adversarial distance for machine learning classifiers [2.2120851074630177]
  This paper investigates the estimation of the more informative adversarial distance using iterative adversarial attacks and a certification approach. We find that our adversarial attack approach is effective compared to related implementations, while the certification method falls short of expectations.
  arXiv  Detail & Related papers  (2024-09-05T14:57:01Z)
• UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection [18.25576487115016]
  This paper focuses on Human-Object Interaction (HOI) detection. It addresses the challenge of identifying and understanding the interactions between humans and objects within a given image or video frame. We propose a novel approach textscUAHOI, Uncertainty-aware Robust Human-Object Interaction Learning.
  arXiv  Detail & Related papers  (2024-08-14T10:06:39Z)
• GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features [68.14842693208465]
  GeneralAD is an anomaly detection framework designed to operate in semantic, near-distribution, and industrial settings. We propose a novel self-supervised anomaly generation module that employs straightforward operations like noise addition and shuffling to patch features. We extensively evaluated our approach on ten datasets, achieving state-of-the-art results in six and on-par performance in the remaining.
  arXiv  Detail & Related papers  (2024-07-17T09:27:41Z)
• xSemAD: Explainable Semantic Anomaly Detection in Event Logs Using Sequence-to-Sequence Models [1.6713531923053913]
  This work addresses a gap in semantic anomaly detection, which typically indicates the occurrence of an anomaly without explaining the nature of the anomaly. We propose xSemAD, an approach that uses a sequence-to-sequence model to go beyond pure identification and provides extended explanations. Our experiments demonstrate that our approach outperforms existing state-of-the-art semantic anomaly detection methods.
  arXiv  Detail & Related papers  (2024-06-28T09:06:52Z)
• Learn Suspected Anomalies from Event Prompts for Video Anomaly Detection [16.77262005540559]
  A novel framework is proposed to guide the learning of suspected anomalies from event prompts. It enables a new multi-prompt learning process to constrain the visual-semantic features across all videos. Our proposed model outperforms most state-of-the-art methods in terms of AP or AUC.
  arXiv  Detail & Related papers  (2024-03-02T10:42:47Z)
• Reimagining Anomalies: What If Anomalies Were Normal? [21.480869966442143]
  We introduce a novel explanation method that generates multiple counterfactual examples for each anomaly. A counterfactual example is a modification of the anomaly that is perceived as normal by the anomaly detector. The method provides a high-level semantic explanation of the mechanism that triggered the anomaly detector, allowing users to explore "what-if scenarios"
  arXiv  Detail & Related papers  (2024-02-22T11:56:44Z)
• AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [59.08735812631131]
  Anomaly inspection plays an important role in industrial manufacture. Existing anomaly inspection methods are limited in their performance due to insufficient anomaly data. We propose AnomalyDiffusion, a novel diffusion-based few-shot anomaly generation model.
  arXiv  Detail & Related papers  (2023-12-10T05:13:40Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• Open-Vocabulary Video Anomaly Detection [57.552523669351636]
  Video anomaly detection (VAD) with weak supervision has achieved remarkable performance in utilizing video-level labels to discriminate whether a video frame is normal or abnormal. Recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to detect unseen anomalies given seen anomalies and normal videos. This paper takes a step further and explores open-vocabulary video anomaly detection (OVVAD), in which we aim to leverage pre-trained large models to detect and categorize seen and unseen anomalies.
  arXiv  Detail & Related papers  (2023-11-13T02:54:17Z)
• Anomaly Heterogeneity Learning for Open-set Supervised Anomaly Detection [26.08881235151695]
  Open-set supervised anomaly detection (OSAD) aims at utilizing a few samples of anomaly classes seen during training to detect unseen anomalies. We introduce a novel approach, namely Anomaly Heterogeneity Learning (AHL), that simulates a diverse set of heterogeneous anomaly distributions. We show that AHL can 1) substantially enhance different state-of-the-art OSAD models in detecting seen and unseen anomalies, and 2) effectively generalize to unseen anomalies in new domains.
  arXiv  Detail & Related papers  (2023-10-19T14:47:11Z)
• ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models [65.79770974145983]
  ASSERT, Automated Safety Scenario Red Teaming, consists of three methods -- semantically aligned augmentation, target bootstrapping, and adversarial knowledge injection. We partition our prompts into four safety domains for a fine-grained analysis of how the domain affects model performance. We find statistically significant performance differences of up to 11% in absolute classification accuracy among semantically related scenarios and error rates of up to 19% absolute error in zero-shot adversarial settings.
  arXiv  Detail & Related papers  (2023-10-14T17:10:28Z)
• Towards General Visual-Linguistic Face Forgery Detection [95.73987327101143]
  Deepfakes are realistic face manipulations that can pose serious threats to security, privacy, and trust. Existing methods mostly treat this task as binary classification, which uses digital labels or mask signals to train the detection model. We propose a novel paradigm named Visual-Linguistic Face Forgery Detection(VLFFD), which uses fine-grained sentence-level prompts as the annotation.
  arXiv  Detail & Related papers  (2023-07-31T10:22:33Z)
• Manifold-Aware Self-Training for Unsupervised Domain Adaptation on Regressing 6D Object Pose [69.14556386954325]
  Domain gap between synthetic and real data in visual regression is bridged in this paper via global feature alignment and local refinement. Our method incorporates an explicit self-supervised manifold regularization, revealing consistent cumulative target dependency across domains. Learning unified implicit neural functions to estimate relative direction and distance of targets to their nearest class bins aims to refine target classification predictions.
  arXiv  Detail & Related papers  (2023-05-18T08:42:41Z)
• Towards Interpretable Anomaly Detection via Invariant Rule Mining [2.538209532048867]
  In this work, we pursue highly interpretable anomaly detection via invariant rule mining. Specifically, we leverage decision tree learning and association rule mining to automatically generate invariant rules. The generated invariant rules can provide explicit explanation of anomaly detection results and thus are extremely useful for subsequent decision-making.
  arXiv  Detail & Related papers  (2022-11-24T13:03:20Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• Improving robustness of jet tagging algorithms with adversarial training [56.79800815519762]
  We investigate the vulnerability of flavor tagging algorithms via application of adversarial attacks. We present an adversarial training strategy that mitigates the impact of such simulated attacks.
  arXiv  Detail & Related papers  (2022-03-25T19:57:19Z)
• X-MAN: Explaining multiple sources of anomalies in video [25.929134751869032]
  We show how to build interpretable feature representations suitable for detecting anomalies in video. We also propose an interpretable probabilistic anomaly detector which can describe the reason behind it's response. Our method competes well with the state of the art on public datasets.
  arXiv  Detail & Related papers  (2021-06-16T15:25:50Z)
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [74.05906222376608]
  We propose adversarial self-supervision UDA (or ASSUDA) that maximizes the agreement between clean images and their adversarial examples by a contrastive loss in the output space. This paper is rooted in two observations: (i) the robustness of UDA methods in semantic segmentation remains unexplored, which pose a security concern in this field; and (ii) although commonly used self-supervision (e.g., rotation and jigsaw) benefits image tasks such as classification and recognition, they fail to provide the critical supervision signals that could learn discriminative representation for segmentation tasks.
  arXiv  Detail & Related papers  (2021-05-23T01:50:44Z)
• Towards Fair Deep Anomaly Detection [24.237000220172906]
  We propose a new architecture for the fair anomaly detection approach (Deep Fair SVDD) We show that our proposed approach can remove the unfairness largely with minimal loss on the anomaly detection performance.
  arXiv  Detail & Related papers  (2020-12-29T22:34:45Z)
• Manifolds for Unsupervised Visual Anomaly Detection [79.22051549519989]
  Unsupervised learning methods that don't necessarily encounter anomalies in training would be immensely useful. We develop a novel hyperspherical Variational Auto-Encoder (VAE) via stereographic projections with a gyroplane layer. We present state-of-the-art results on visual anomaly benchmarks in precision manufacturing and inspection, demonstrating real-world utility in industrial AI scenarios.
  arXiv  Detail & Related papers  (2020-06-19T20:41:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.